Friday, June 29, 2018

SMS over IRC

A couple years ago, I moved from Google Voice to custom infrastructure.  That system used XMPP to send SMS.  Now that I've moved from XMPP to IRC, I needed a new set up for sending SMS.  This post is about a small proxy I wrote to translate between SMS and IRC protocols.

When I'm connected to our family IRC server, I can send a message to any nick that's a valid phone number.  The server converts that message into an HTTP request to Twilio to send an SMS to that number.  I map our IRC nicks to our phone numbers to correctly set outgoing caller ID.

Some of my extended family (mom, dad, siblings, etc.) aren't connected to our IRC server.  I contact them frequently enough that I didn't want to use their phone numbers in IRC, so I've given them aliases.  The server pretends that they're on IRC under those aliases.  For instance, when I send a privmsg to the IRC nick "mom", it converts that into a Twilio API call that sends an SMS to my mom's phone number.  As far as my IRC clients are concerned, my mom is on IRC.

When sending SMS, Twilio provides helpful status updates via HTTP.  I've translated some of those into IRC messages too.  In an IRC client, it looks like this:

[12:01:02] michael: Any big plans for your birthday?
[12:01:03] mom: ⌛
[12:01:07] mom: ✓
[12:05:22] mom: I'm going to Cirque du Soleil with some friends


The first two emoji are synthetic messages generated by the IRC server.  ⌛ means, "Twilio has successfully delivered the message to the recipient's phone company" and ✓ means, "the phone company delivered the message to the recipient's phone".

Incoming SMS work as you'd expect.  The phone number to which the SMS was sent, determines which IRC user receives the corresponding privmsg.  If the phone number has an alias, that alias is used as the IRC nick; otherwise, the phone number is used as the nick.

It all works really well in practice.  I can pretty much pretend that everyone I care about is on IRC all the time.

Twilio doesn't support group SMS, which is too bad.  I'd love to translate those into ad-hoc IRC channels.  Twilio supports MMS, but I receive them so rarely that I haven't added IRC support yet.

The code is available, in case you're interested.  It's a couple hundred lines of Go.

Thursday, June 28, 2018

Dadurday

As my five kids have gotten older, I've noticed the world and its activities encroaching more and more on our family time.  While we were traveling in Europe, we changed cities every two weeks and had no outside obligations, so our family spent lots of time together.  It was great.  Our friendships strengthened and we grew closer.

As we prepared to return home, my wife and I decided that we would establish the first and third Saturdays of each month as "Dadurday".  A day when the kids and I would block everything else out and just spend time together: no chores, no outside obligations.  So far we've watched a movie, played board games, read stories, watched the World Cup, collected coal to try to start a fire, had sword fights, and gone on walking adventures.  The kids seem excited about the whole thing and are constantly suggesting activities for the next Dadurday.

The outside has tried to occupy these days, as we suspected it would.  I suppose it's a corollary to Parkinson's Law that non-family activities expand to fill all available time.  Fortunately, Dadurday was on the calendar first.  When other activities try to crowd in, we politely decline and say that we have a previous obligation.  Just knowing that these unstructured, playful days with my kids are waiting for me every couple weeks makes me smile.

Of course, my kids have friends and other interests.  Most of the time they can be scheduled away from Dadurday, but sometimes there's a conflict.  As a family, we decided that each kid gets 3 exemptions per year that let them skip Dadurday if they want to do something else that day.  So far three kids have spent four exemptions total, all to attend birthday parties.

Between Dadurday and our family night on Mondays, I suppose that some people would think it's too much time together, but it's a great balance for us.

Tuesday, June 12, 2018

Goodbye XMPP. Hello IRC

Up until 2 years ago, my wife and kids and I stayed in touch with Google Hangouts.  I disliked being dependent on a third party, being unable to program the system to match our needs, and not having access to it on all the devices we care about, so we switched to XMPP.  It was an improvement and has served us well for the last couple years.

A couple months ago, we started seeing large delays in message delivery times.  We were also getting extended, repeated disconnects from the XMPP server.  Our messaging stack is selfhosted and open source, so I prepared for a debugging session.  As I got ready to dive in, I had a nagging feeling that there was way too much code at play here.  We ran Prosody, which is a great XMPP server, but it's 35k lines of code.  My SMS to XMPP gateway adds another 1k.  All we wanted to do was send small pieces of text between 7 people and set up some chat rooms.  That much code felt like overkill.

After a few hours work and 500 lines of Go, I had a custom IRC server with a built in BNC.  With another 200 lines, I had a working SMS to IRC gateway, so I can send and receive SMS from my IRC clients.  Obviously, setting up chat rooms was painless.

We had some minor glitches during initial deployment, but because the whole system is so small, it was a cinch to debug and repair.  It's been really solid since then.  The minimalism is refreshing.

The code is available in case you're interested. It won't compile by itself since it's part of a larger family server (git, movies, personal assistant, Asterisk dialplan, etc).

Wednesday, May 09, 2018

Easier random numbers in Go

Abstract: Replace import "math/rand" with import "github.com/mndrix/rand" and get high-quality random numbers every time.

Like many Go developers, I use Go's math/rand package for generating random numbers.  That package defaults to a predictable stream of outputs, so you have to seed the generator if you actually want random numbers. I'm embarrassed to admit how often I've forgotten to seed the generator.  In large projects, I see the generator being seeded multiple times across different files because everyone wants to be certain the generator has been seeded.

Even if I do remember to set a seed, I often get hung up on other questions like:
  • is my seed good enough?
  • are these random numbers high enough quality for my application?
  • do I need to reseed the generator after a while?
  • how often?
I just want random numbers without all this complexity.

More than 20 years ago, OpenBSD developers were seeing similar problems.  In 1996, they addressed it by introducing arc4random.  You call the function and get a high-quality random number every single time.  There's no seeding, no failure mode, no need to consume file handles, just random numbers.  It's refreshingly simple.

I wanted that same simplicity in Go, so I wrote github.com/mndrix/rand.  It's just a thin wrapper around Go's crypto/rand which gives those high-quality numbers the same clean interface as math/rand.  In many cases, you can just replace import "math/rand" in your code with import "github.com/math/rand" then delete your calls to math.Seed() and everything works.

Saturday, March 17, 2018

World tour: Morocco

Since leaving Hungary two months ago, we've been in Morocco.  I tried to visit North Africa sixteen years ago during a trip to France and Spain, but was thwarted at the last minute by some trouble with a train conductor.  That near miss made me all the more excited for Morocco on this trip.

We started in Marrakesh near the base of the Atlas Mountains.  I knew there were mountains in Morocco, but I had no idea they were so large.  The tallest mountain in the range is 13,670 feet above sea level and Marrakesh is at 1,500 feet so the mountains have a larger prominence than those along the Front Range or the Wasatch Front.  The Atlas were beautiful and covered with snow.  We didn't bring enough clothes to stay warm in the mountains in February, but I would love to come back to Morocco and hike all over the Atlas someday.

The average wage in Morocco is quite low, so we found it affordable and convenient to hire a private van to drive us between cities instead of using the trains.  The highway system in Morocco is modern and well maintained.  Their rest stops are some of the best I've seen.  Each one had a selection of food vendors, at least one good playground and a small mosque where travelers could pray.  If we visit Morocco again, we'll definitely rent a car.

Anyway, Casablanca was next in line.  Our apartment there was just a couple blocks away from the tallest mosque in the world.

The mosque was really pretty.  It reminded me how much I like Arab architecture with its clean lines and subdued, natural colors.  The mosque was surrounded by quiet gardens:

Between the gardens were covered walkways.  Each one had ceilings with hand carved and hand painted decorations:

Other than the mosque, Casablanca, like most Moroccan cities, is not a destination for sightseers.  Unlike Europe, there isn't much of historical or architectural interest.  We were quite happy with that.  We just enjoyed walking through the city and living our life alongside everyone else.

After Casablanca, we went all the way north to Tangier.  Our apartment was on a hill overlooking the Strait of Gibraltar.  It was cool to see Spain on the other side and to watch all the container ships crossing between the Atlantic and the Mediterranean.  It rained almost the entire time we were there, but we managed one day at a nearby beach once the sun came out:

Our last stop was Fes.  Our apartment was inside the medina right next to the souq.  We went into the souq each day to buy groceries for that day.  There are hundreds of vendors, each with a small shop specializing in one or two items.  You buy bread at one place, eggs at another, fruit at yet another, etc.  It's sort of like a farmer's market the size of an entire city.  By the end, Kinsey was a pro and knew which shops had the best food of different kinds.  The fruit vendor that she preferred recognized our children as they walked through the souq and would give them free strawberries as they passed by.

Overall, I liked Morocco.  If we visit again someday, I'd try to spend most of our time in the rural areas.  The cities are a fun experience once, but the beautiful mountains and open spaces called my name.  I would also love to cross the Atlas and spend some time on the Saharan side of the country.  There's so much beautiful world and so little time.

Next stop: New York and Las Vegas

Thursday, February 08, 2018

World tour: Bulgaria & Hungary

After Jerusalem, our next stop was Sofia, Bulgaria.  I had been looking forward to this stop because it would be my first extended visit to the Eastern Bloc since serving as a missionary in the former East Germany.  After all the warm temperatures around the Mediterranean, we were also excited for some cold weather.  Bulgaria did not disappoint.  The mountains around the city were beautiful and covered in snow.  Sofia has some huge, forested city parks which were great for walking on cold days.  Despite the cold, Haven insisted on doing laundry in shorts and bare feet:


And of course, the cold didn't prevent our customary visits to every nearby playground:


I loved it in Bulgaria.  The scenery was beautiful, the people were helpful when asked but mostly kept to themselves, everything cost half of what it does in America.  It was also fun to practice the Cyrillic alphabet.  There were no tourists anywhere and the whole country had a quiet, modest and unassuming feel.  I would definitely go back.

Oh, we also did our first escape room as a family.  Copyright laws don't seem to be enforced much in this part of the world, so ours had an Indiana Jones theme.  Honestly, it was more entertaining than the last film approved by Lucasfilm.  Fan fiction for the win.


As we left Bulgaria to fly to Hungary, we got held up at passport control.  The passport agent didn't believe that we had five children and insisted that Kinsey was too young to be their mother.  She kept checking Kinsey's birth date against the children's birth dates and asking the children how old they were.  She was friendly about the whole thing, just being cautious about child trafficking I suppose.

I don't have much to say about Budapest.  It was a busy, modern European city.  It had some cool, old buildings and the Danube was pretty.  It also has the best public transit I've ever seen: clean, fast, frequent trains, always on-time.  Budapest also has a really good model train museum: Miniversum.  The Parliament building was also nice:


Some random observations from our trip so far:

First, we've used six different ride-hailing apps in nine different countries.  Uber works in some but not in others.  Each country has its own app for connecting with the local taxi network.  This industry is ripe for consolidation.  Someday, someone will come along, buy up all the little guys, polish up the experience and make lots of money.  TaxiMe in Bulgaria was one of my favorite ride-hailing apps.  They implement surge pricing through a real time auction.  If you request a ride and don't get a cab, you can place a bid higher the standard fare to lure a taxi.  All passengers in the vicinity are bidding against each other to win a taxi's attention.  It was basically a double auction with TaxiMe acting as auctioneer and it was fun.  I found myself wishing the taxis were busy so I could play the "game".

Second, American credit card companies can be overzealous with anti-fraud measures when traveling in the Middle East or Eastern Europe.  We rarely have trouble paying in-person with a chip card, but are almost always refused when paying online.  On several occasions, I've had to route my Internet traffic through one of my US servers to circumvent the anti-fraud measures so that I can complete a transaction.  So their "security" is not very secure, but it does restrict foreign credit card transactions to only the tech-savvy.  Having a small wad of local cash has proven helpful in many instances.  Card companies give better exchange rates, but cash is handy as backup.

Next stop: Morocco

Saturday, January 13, 2018

World tour: Levant

After leaving Athens, our world tour continued in the Levant: a brief stop in Beirut followed by two weeks each in Jordan and Israel.  A couple months ago, we happened to be in Spain during Catalan independence.  This time we were in the Levant just as the US recognized Jerusalem as Israel's capital.  It made the politics of the region especially obvious.

Here's Jericho on the terrace of the BYU Jerusalem Center describing some of Jerusalem's history. The Dome of the Rock is in the background.


Anyway, our flight from Athens to Amman was on Middle East Airlines through its main hub in Beirut.  We only spent 4 hours in the airport in Lebanon, but it was a fun introduction to the region.  There were European tourists wearing jeans and sweats, Lebanese business men wearing Italian suits, and Saudi women wearing abaya with niqab.  Signs were in Arabic or French with very little English and numbers all used the eastern Arabic system: ٠١٢٣٤٥٦٧٨٩.  Fortunately, the airport gates were all labeled with western Arabic numerals so we knew where to go.  The gate signs also served as a Rosetta stone so we could decipher eastern Arabic numbers.

Our flight path from Beirut to Amman is a good example of how this region's politics impact people's daily lives.  The distance from Beirut to Amman is only about 130 miles, as the crow flies.  Israel won't allow Lebanese planes to fly through their airspace.  Syria is in the middle of a civil war so they can't fly over Damascus either.  Instead, our flight flew over the Mediterranean to Tripoli then east through the mountains and deserts of Syria to cross Jordan's border.  Then they can fly freely into Amman.  On the positive side, the circuitous route gave us time to eat our in-flight meal:


I really liked Jordan.  It's only the second place on our trip so far to which I hope to return someday (Iceland being the other).  After arriving at the airport, we got a taxi to our apartment.  The driver of the small taxi assured us that all seven of us and our luggage could fit into his little taxi.  We managed to squish together with me in the front passenger seat and the other six of us on laps in the back seat.  As we got under way, the driver made known in very broken English and with a broad smile that if the police saw us, he would get in trouble so he expected us to pay the fare of two taxis instead of just one.  I smiled at his bargaining and assured him that we'd pay him a reasonable fare.  When we got to the apartment, I paid him what I had budgeted for the trip and he accepted with another big smile.  It was only about $40 and about half what we'd paid for two airport taxis in other countries but roughly double what he expected.

For the rest of our stay in Amman, we used Uber.  It's illegal in Jordan (driver subject to $200 fine and loses his car for two weeks), but in practice nobody seems to care.  Uber can get you almost anywhere in the city for $3.  The police almost caught us riding an Uber from the mall one day, but the driver had lived in Los Angeles and told the officer that we were his American friends.  He spent the rest of the trip telling me how much he liked America and how he hoped to return.  I assured him that we'd love to have him in our country again.

Bedouin are still prominent in Jordan.  For example, every morning a Bedouin man brought his sheep and goats to graze in the field across the street from our apartment.
Nearby restaurants would dump old bread into the field during the night and his sheep and goats would eat it the next morning.  When they were done, he'd move them elsewhere in the city to graze. Throughout the city, there are Bedouin living in tents and herding goats right next to luxury apartment buildings and shopping malls.

We visited Petra, which is really amazing.  The whole area looks like the red rocks of southern Utah, but the rocks are carved into buildings and decorative caves.

We planned to see more of Jordan, but half of us got colds and sore throats so we had to lay low for most of a week.  This is the first real illness we've had on the trip, which is great.

A few more random observations about Jordan:  The large shopping centers all have metal detectors as you walk in. There is always a male and a female security guard working together.  Male guests work with the male guard and female guests work with the female guard.  Jordan is a very modern and western feeling country, but you can clearly see the deference to Muslim modesty throughout the culture.  I loved hearing the Adhan (call to prayer) five times each day.  It was even played inside the grocery store so that everyone could hear it.  I expected people to drop everything and go pray, but they didn't.  Islam gives substantial leeway in when the prayers are performed so most people finish what they're doing and then go somewhere private to pray.  The large shopping centers all have dedicated prayer rooms so that guests don't have to leave the building to pray.  Oh, and spices, lots and lots of spices:


Jordan's pharmacy system is also really nice.  Pretty much everything is available without a prescription.  After being away from the States for three months, we'd run out of certain medications.  We tried to restock in Spain, Italy and Greece but they all required a prescription.  The Jordanian pharmacy gladly sold us what we asked for without questions.  The people in Jordan are also exceptionally hospitable.  When we arrived at our apartment, the landlord offered to walk 2-3 blocks with me to help me purchase a SIM card so one of our phones would work.  When I asked one driver if there was anywhere nearby to buy some lunch for the kids, he drove to an amazing schawarma restaurant, accompanied me inside, made some menu recommendations, translated for me as I ordered and walked me through the process of picking up the food from elsewhere in the restaurant.  The whole country was like this.  Most people spoke some English, but the ones who didn't all knew how to say "Welcome to Jordan".  About 30% of the country's population is refugees from elsewhere: Palestine, Iraq, Syria.  Welcoming people is a deep part of their culture.  They also like American movies.  We watched Star Wars: The Last Jedi in a beautiful theater in Amman.  The theater had assigned seats, like at an NFL game and Arabic subtitles, unlike an NFL game.


Depending on who you ask, Jordan has the second or third fewest water resources per capita of any country on earth.  In Amman, the water pipes are turned on once each week.  Each apartment has large water tanks on the roof which they fill up when the pipes turn on.  Not being familiar with the scarcity, we accidentally used up all our water 24 hours before the next delivery.  Our landlord graciously moved some water from another apartment in the building to help us out.  If there hadn't been water available in the building, you can hire private water trucks which can refill your tanks at market prices.  Quick economic tangent: as with most resource scarcities, the water scarcity in Jordan is caused by restrictions on the price system.  Water provided by the city costs 1/25 of the market price as charged by private delivery trucks.  The delivery trucks never run out of water, only the city.

Anyway, enough about Jordan.  From Amman we crossed the King Hussein Bridge into Israel.  The process usually takes about 4 hours.  We got there in time for the first bus of the day and there weren't many tourists so we crossed in only 2 hours (for a 3 mile drive; thanks politics).  The Jordanian side of the transfer was relaxed and disorganized.  The Israeli side was organized and strict.  Both sides were friendly.



In Jerusalem, we visited Yad Vashem (a Holocaust musem), the Garden of Gethsemane (pictured above), Mount of Olives and the Old City.  The holy sites are so crowded and touristy that I can't really recommend them.  Some people have a really spiritual experience in these places, but they weren't for me.  Walking around the Old City was cool though.  It was fun to see all the Bar Mitzvah celebrants walking through the city gates.  Gideon found a paper map of the Old City in our apartment, so he guided us during our explorations:

The Old City also has some nice squares and quiet little corners where you can just enjoy the ambiance:

Unlike Jordan, everything in Israel is really expensive.  Food and taxis are almost 10x more expensive on the Israeli side of the border.  It's no worse than New York City or Boston, but it was a bit of a shock after Jordan.  Random tidbit: burger joints in Jerusalem don't sell cheeseburgers because Orthodox Jews don't eat meat and dairy together.

My favorite thing about Israel is probably Hebrew.  I'm kind of a language nerd, so Hebrew was really fun.  Prior to the mid 1800s, Hebrew was pretty much a dead language.  It was only used for Jewish liturgy and spiritual purposes, but had no native speakers or use in modern life.  Now it's spoken natively by millions of people and the primary language of commerce throughout Israel.  Hebrew is the only example of a natural language being resurrected from the dead like that.  It was cool to drive through Jerusalem and Tel Aviv and see the scale of the success of that linguistic experiment.  Since the country favors Hebrew so strongly, there was almost no signage in English.

Both Israel and Jordan have substantial restrictions on the free practice of religion.  The Church of Jesus Christ of Latter-day Saints has church units in both countries, but they mostly accommodate foreigners (State Department employees, students, etc).  Most locals are prohibited from participating, so meetings are done in English instead of the local language.  Amman has an Arabic branch since Jordanian Christians are allowed to convert to a religion of their choice.  Between Athens, Amman and Jerusalem, we had five consecutive weeks where we understood what people were saying to us at church.  In Jordan, our church meetings were on Friday (out of respect for the Muslim holy day) and in Israel they were on Saturday (ditto for Jewish shabbat).

It's also been interesting to see the extent to which English has become the world's lingua franca.  Our flight from Tel Aviv to Sofia had a Hungarian crew with mostly Israeli and Bulgarian passengers, but everyone on board was communicating in English.  It's more true than ever, "the world is flat."

Next stop: Bulgaria and Hungary.