Monday, July 17, 2017

Parenting? Roll a Dice

The Problem

I have five children between the ages of 13 and 3.  Everywhere we go, there's a scarce resource that all the kids want.  Maybe it's an elevator button that everyone wants to push.  Maybe it's a favorite spot in the car where everyone wants to sit.  On the flip side, there are chores that nobody wants to do: take out the trash, load the dishwasher, etc.  I used to make assignments round-robin, trying to give each kid a turn at fun and chores.  I've tried other scheduling algorithms too, but I inevitably forget whose turn it is and then everyone tries to straighten me out: "But it was Gideon's turn last time. No, that was Haven not me. ..."

I've tried for years to convince them that these things don't matter and that life is not fair.  I'm apparently poor at communicating that message.  So I decided to take my own sermon to heart:

My Solution

If it really doesn't matter, why not pick kids at random?  I installed Random Name Picker on my phone.  I created a list named Children and added each child's name to that list.  In the list settings, I chose "With replacement" and set "# of names chosen" to 5.  When I click Choose, the app shuffles all the children into a random order.  Now that little Choose button resolves all trivial, family disputes.  We've been doing this for months and it works great.

Only space for one kid to help Dad at the store? Click Choose and the top name wins.  Choosing 2 ice cream flavors at the store? Click Choose and the top two names win.  On vacation? Click Choose in the morning and read everyone the full list to assign priority for pushing elevator buttons that day.  Assigning all children a random priority has proven especially helpful.  Each kid remembers their place in line.  No matter which subset of children happens to travel in the elevator at any given time, they instantly know whose turn it is to push the buttons.

One extra rule proved useful: Dad always clicks Choose.  Otherwise, things get meta really fast: "I get to click Choose to see who gets to click Choose to see who gets to pick the ice cream"

Sunday, July 02, 2017

Goodbye Duolingo

Today I'm giving up a 665 day streak in Duolingo.  I've removed the app from my phone.  Duolingo is still a helpful tool for many and they have potential to become great again.  Unfortunately, the trend in their recent changes has encouraged me to leave.  I'm mostly writing this to warn myself about practices that can drive customers away.

I didn't mind when Duolingo started showing ads.  I knew they had to do it eventually.  They even did a good job of showing ads in predictable locations and using ads of relatively high quality.  I anticipated a subscription model allowing me to pay $1 per month to remove the ads and expected no other functionality for my subscription.  Unfortunately, the subscription costs 10x more than I was willing to pay.  For reference, Duolingo charges $9 per month.  Netflix charges $7 per month and provides much greater value.

No big deal.  I'll skip the subscription and just look at the ad after each lesson.  I'm sure that 90% of users do the same thing.  Duolingo doesn't seem content with that, so they randomly move the location of the "close this ad" button on each page.  Sometimes it's in the top left corner.  Sometimes it's in the lower right corner.  This inconsistency is disrespectful.  It's playing games with my time in an apparent attempt to manipulate me into clicking their ads or subscription button.  If most users are going to skip the ads, the "close this ad" button should be predictable and convenient (on the bottom where a thumb can quickly reach it).

It was annoying, but I was willing to live with it.  Duolingo has to make money.  They're being a little pushy about it, but I can overlook that.  Then they changed the "strengthen my skills" button so that it only performs a timed practice.  I thought, "maybe they've seen that this produces better results, so I'll try it".  After using it for 2-3 weeks, I found myself skipping any question which took longer than 1-2 seconds to answer (listening comprehension, free text response).  The new rules of the game were encouraging me to learn a language less effectively.  For people whose focus is conversational speech, it might make sense to practice under time pressure.  My goal has always been reading comprehension.  In that context, time is not an issue.  I can spend as much time as I want on a German Wikipedia page and nobody will care.

Android has always been a second-class citizen for Duolingo.  Useful features come to iPhone first and often don't make it to Android for months or years.  It's no big deal since the core functionality is available on Android.  Duolingo makes it very clear that they're trying to hire an Android developer.  They've been trying for as long as I can remember.  A couple days ago, I installed an update to the Android app.  Even though I only use the app for 10 minutes each morning, it now consumes 9% of my battery each day.

In a market for user attention that's so competitive, you really have to play at the top of your game.  Anything pushy or disrespectful can have large costs.  Repeated minor issues like this can push a user away.  I hope I can remember that as I work on software.

Duolingo.  Thanks for 665 days of fun.  Sorry that things didn't work out between us.

Monday, June 19, 2017

Toggle syntax highlighting to catch bugs

I usually write code with syntax highlighting enabled.  While preparing my final commit message, I view the proposed diff in its own color scheme (red for removed lines, green for added lines, white for context lines).  Even though I've spent hours working on a patch, I often spot mistakes in my newly added lines as soon as the color scheme changes.  Apparently, psychologists already knew about this phenomenon: “Once you’ve learned something in a particular way, it’s hard to see the details without changing the visual form.”  The article suggests other visual changes like using a different font or printing to paper.

I use two other, related hacks for helping myself find mistakes in my code:

  • go to sleep and review my code in the morning
  • watch a video, play a game, read a book, work on a completely unrelated problem to force my mind to lose as much of its mental model as possible then review my code again
In each case, the new perspective often reveals details that I overlooked before.

Saturday, May 20, 2017

Sensitive survey questions

Do you steal from your employer? Do you lie on your taxes? Have you cheated on your wife?  If you want to gather statistical information about these questions, you can't ask directly.  Most respondents will lie.  I'm aware of three methods for addressing the problem, two of them are quite clever.


Bogus Pipeline

The first one is not particularly clever.  Hook the subject to a machine.  Tell them it's a lie detector even though it's not. Ask them to respond honestly and pose a few baseline questions to which you know the answer (What's your name? What day is it? etc). After each answer, have the machine indicate that it detected truth.  Now ask the subject to respond deceptively and ask more baseline questions. After each response, have the machine indicate that it detected a lie.  Now hide the machine's truth/lie indicator and ask your questions.  Most subjects will tell the truth.

This is called a bogus pipeline. It's complicated to implement, requires physical access to the subject and not as accurate as other techniques.

Randomized Response

Ask the subject to flip a coin but don't tell you what it is.  If it's heads, they should answer truthfully. If it's tails, they should answer yes (or whatever the socially unfavorable answer is).  Now ask your question.  Applying some simple math to the aggregate responses, you can accurately calculate the percentages you want to know.

This one's pretty helpful, but it requires the subject to have a coin (who uses coins anymore?).  The subject must also be smart enough to recognize that the coin gives him deniability. It seems obvious, but it's not obvious to everyone.

Unmatched Count

Construct an innocuous survey along these lines: "How many of the following statements are true about you? I own a dog. I drink coffee. I've been married. I have brown hair."  Construct a second survey, identical to the first but add your sensitive statement, "I cheat on my taxes".  For each subject, randomly give them one survey or the other.  Calculate the average answer for each type of survey.  The difference between the two averages tells you the percentages you want to know.

This one's my favorite. Since the subject only tells you their final count, it's obvious to them that they've divulged no sensitive information.  The math for analyzing the results is similarly easy.

Do you know of any other techniques?

Tuesday, May 02, 2017

Switching to OpenBSD

Short story:

After 12 years, I switched from macOS to OpenBSD.  It's clean, focused, stable, consistent and lets me get my work done without any hassle.

Long story:

When I first became interested in computers, I thought operating systems were fascinating. For years I would reinstall an operating system every other weekend just to try a different configuration: MS-DOS 3.3, Windows 3.0, Linux 1.0 (countless hours recompiling kernels).  In high school, I settled down and ran OS/2 for 5 years until I graduated college. I switched to Linux after college and used it exclusively for 5 years. I got tired of configuring Linux, so I switched to OS X for the next 12 years, where things just worked.

I was pretty happy with OS X.  It gave me Unix and mostly got out of the way so that I could write software.  I wrote about enjoying Apple's simplicity.  Snow Leopard even spent an entire release cycle just fixing bugs and improving performance.

But Snow Leopard was 7 years ago. These days, OS X is like running a denial of service attack against myself.  macOS has a dozen apps I don't use but can't remove. Updating them requires a restart.  Frequent updates to the browser require a restart.  A minor XCode update requires me to download a 4.3 GB file.  My monitors frequently turn off and require a restart to fix.  A system's availability is a function of mean time between failure and mean time to repair.  For macOS, both numbers are heading in the wrong direction for me. I don't hold any hard feelings about it, but it's time for me to get off this OS and back to productive work.

So where do I go now?  We own 5 Chromebooks and they have great availability.  Updates are infrequent, small, fast and nearly transparent.  Unfortunately, I need an OS where I can write and compile code.  I also want it to run on older, commodity hardware so I can replace a broken laptop for $400 instead of $2,000.

I considered several Linux distributions.  Lubuntu seemed promising, but it was too bloated for my taste.  A couple years ago, I tried Ubuntu on a Dell XPS Developer Edition for a few months.  Even with hardware designed for Linux, it was too fragile. Desktop Linux has also become even more complex than when I used it a decade ago.  I just want to get my work done, not feed and maintain an OS.

I was reminded of OpenBSD during the Heartbleed scare.  While everyone else was complaining about OpenSSL and claiming that open source had failed, the OpenBSD developers quietly drew their machetes and hacked out hundreds of thousands of lines of bad code, forking off LibreSSL where they can keep it clean and stable.  The OpenBSD community is like that: focus on what's really important, hold your code to a high standard, ignore all the distractions.  They're not trying to live in the past, just trying to make the future a place worth living.

Anyway, I found OpenBSD very refreshing, so I created a bootable thumb drive and within an hour had it up and running on a two-year old laptop.  I've been using it for my daily work for the past two weeks and it's been great.  Simple, boring and productive.  Just the way I like it.  The documentation is fantastic.  I've been using Unix for years and have learned quite a bit just by reading their man pages.  OS releases come like clockwork every 6 months and are supported for 12.  Security and other updates seem relatively rare between releases (roughly one small patch per week during 6.0).  With syspatch in 6.1, installing them should be really easy too.

I also enjoy that most things are turned off in OpenBSD by default.  The base installation is sparse.  It assumes that I'll enable a service or install a tool if I want it.   So I'm not constantly facing updates for software I never use.

My experience with OpenBSD is still young, but I really like what I see so far.

Thursday, April 06, 2017

Using Project Fi

I signed up for Project Fi last month.  It's been a real pleasure to use.  I expected it to be a step down compared to Ting, but I was wrong.  For my use cases, Fi is slightly better in a couple ways.  Fi's customer support isn't as good, but it's acceptable and better than most phone companies.

Billing


Project Fi canceled my Google Voice account during sign up, but it transferred all my GV account credit over to the new account.  The credit wasn't visible on the first bill but appeared on the second.  It was nice not to lose those GV funds.  (Fi also transferred my voicemail greetings and blocked numbers from GV).

I really like the way that Fi charges for data.  I chose the 1 GB plan (since there's no penalty for overages).  Last month I consumed 1.165 GB of data.  On Ting I would have paid $10 for crossing into the second GB.  On Fi, I paid only $1.65 extra; exactly covering my overage.

Fi provides free data-only SIM cards whose usage is just added to your account.  Since my family uses VoIP for all phone calls (either SIP or Google Voice) and SMS, everyone just needs data.  I gave everyone a data-only SIM card and it's been working great.  The SIM cards work in every device we've tried, old and new.  This avoids the $20/month charge per phone line.  Fi billing breaks out usage for each device.  The only downside is that you can only order one data SIM at a time.  It took me a few weeks to place orders for all the cards I needed.

Coverage


Ting is built on T-Mobile's network.  It has great coverage almost everywhere I go.  The one exception is the northwest quarter of my grocery store.  Data signal in that part of the store was always missing.

Since Project Fi automatically switches between T-Mobile, Sprint and US Cellular networks, depending on signal strength, I expected Fi to do better.  It did.  This dead spot is no longer a problem.  In this particular store, Fi often switches to the Sprint network then switches back to T-Mobile when I go elsewhere.  Never underestimate the power of a layer of abstraction.  Data-only SIM cards only use T-Mobile, so their coverage is identical to what I had on Ting.

I use Signal Spy to see which network my phone is currently using.  I find it gratifying to drive through town and watch Fi switch networks.  I look forward to some road trips this summer to see how effective it is in that scenario.

Fi's switching algorithm sometimes sticks with a network whose signal is slightly weaker than the alternatives.  Theoretically this could hurt battery life, but I've never had it impact connectivity.  It does annoy my OCD a little.

Conclusion


Overall, I'm very happy with Project Fi and will probably stick with it after I return from Europe.  I hope Fi's international coverage is as good as the US coverage.

Wednesday, March 29, 2017

Cancel a Stripe subscription in App Engine

Stripe's API for canceling subscriptions requires that you send an HTTP DELETE request.  If you want to cancel the subscription at the end of the current billing period (instead of immediately), you need to include a body parameter at_period_end=true in the request.  That's fine.  RFC 7231 section 4.3.5 says that a DELETE request is allowed to have a body but it "has no defined semantics".

Unfortunately, App Engine's urlfetch service silently removes the body from all outgoing DELETE requests.  Trying to cancel a Stripe subscription in App Engine always does it immediately, even if you asked to cancel at the end of the billing period.   Google's known about the problem since 2008, but never fixed it.  The glitch impacts many APIs other than Stripe.

Fortunately, you can use App Engine's sockets API to construct a workaround.  In Go, you build an HTTP client whose transport uses the socket API to make outbound network connections.  That uses Go's built in support for HTTP DELETE requests and avoids the bugs in App Engine's urlfetch service.

httpClient := &http.Client{
 Transport: &http.Transport{
  Dial: func(network, addr string) (net.Conn, error) {
   return socket.Dial(c, network, addr)
  },
 },
}


Tuesday, March 28, 2017

Programming Languages by Spec Size

I was curious which programming language has the smallest specification.  Which one has the largest?  For each language, I printed the spec to a PDF and counted the pages.  Go is the smallest.  C++ is the largest.

This is a very rough estimate of language complexity since each specification varies in style and purpose.  For example, Prolog includes Annex A which is only informative but doubles the size.  Haskell 2010 includes Part II - Libraries which defines the standard library, not the language.  Anyway, here's the list:

If you'd like me to add other languages, comment with a link to the spec and I'll update this post.

Monday, March 20, 2017

gsutil: No module named google_compute_engine

While trying to run gsutil ls on a Compute Engine VM, I received a stack trace like this:
Traceback (most recent call last):
  ...
ImportError: No module named google_compute_engine
It turns out that gsutil doesn't like Python installed from Linuxbrew.  It really wants to use the system Python.  The following works fine:
env CLOUDSDK_PYTHON=/usr/bin/python gsutil ls ...
Easy enough to fix.  Nobody else had documented trouble with this configuration so here you go Internet.

Wednesday, March 15, 2017

AGI Script completed, returning 4

I use Asterisk to run my phone system.  Most of my dialplan is an AGI script written in Perl.  In certain circumstances, I kept finding the following message in Asterisk's logs:
AGI Script ... completed, returning 4
My script should always return 0, so seeing an exit code of 4 was unexpected.  After this log message appeared, no further code in my AGI script was executed, also unexpected.

After a bunch of debugging, I discovered that Asterisk sends SIGHUP to your script if a caller hangs up.  The exit code 4 was Asterisk's interpretation of Perl's exit code when SIGHUP wasn't handled.  I installed a handler for that signal and now everything's grand.

Prior to tonight's debugging, I had never used the sigtrap package.  Its stack traces were very helpful.

Tuesday, March 07, 2017

Serving Git repositories from Go

I have a small web server written in Go.  I wanted to serve Git repositories from this server.  It turned out to be surprisingly easy since git-http-backend and the cgi package do all the hard work.

First, define a function for handling Git's HTTP requests:

func git(w http.ResponseWriter, r *http.Request) {
        username, password, ok := r.BasicAuth()
        if !ok || username != "john" || password != "secret" {
                w.Header().Set("Content-Type", "text/plain")
                w.Header().Set("WWW-Authenticate", "Basic realm=\"example.com git\"")
                w.WriteHeader(http.StatusUnauthorized)
                fmt.Fprintln(w, "Not authorized")
                return
        }

        const gitRoot = "/path/to/git/repositories/"
        h := &cgi.Handler{
                Path: "/usr/lib/git-core/git-http-backend",
                Root: "/git",
                Dir:  gitRoot,
                Env: []string{
                        "GIT_PROJECT_ROOT=" + gitRoot,
                        "GIT_HTTP_EXPORT_ALL=", // we require auth above
                        "REMOTE_USER=" + username,
                },
        }
        h.ServeHTTP(w, r)
}

Second, connect that function to the mux you're already using:

mux.HandleFunc("/git/", git)

Now I can clone from and push to my little Go server.

Saturday, March 04, 2017

Sharing is dangerous

Last week I tried pass as an open source replacement for 1Password.  It was almost exactly what I wanted.  Unfortunately, it uses GnuPG for encryption.  GnuPG is a pain and bloated (444k lines of code).  Since pass just runs the gpg binary, I wrote a quick script that implements the gpg shell interface but does encryption with a much smaller library.  The proof of concept worked.  I made a note to migrate to pass later and uninstalled it.  I forgot to remove my fake gpg script.

This morning I tried running "brew update".  It fetched Git repositories then stalled without hints about the cause.  After too much debugging time, I discovered that it was running my fake gpg which blocked waiting for input.  I deleted fake gpg then "brew update" proceeded fine.

Too much sharing

The wasted debug time was clearly my fault, but it reminded me how dangerous sharing is.  Too much of our software is a wobbly tower of dependencies.  Sure, you can change the bottom block, but it's risky.

The well-known costs of global mutable state are a symptom of problematic sharing.  Package management becomes NP-complete dependency hell when sharing is mandatory (assumption 4). The recent Cloudbleed vulnerability was mostly a problem because of sharing:

Because Cloudflare operates a large, shared infrastructure, an HTTP request to a Cloudflare web site that was vulnerable to this problem could reveal information about an unrelated other Cloudflare site. (emphasis added)

In other words, FitBit and Uber have a security vulnerability because some random WordPress blog generated bad HTML.

Don't share, copy

Some cultures don't like to share.  Docker containers isolate applications from each other.  Node prefers to load a module that's the least shared.  OpenBSD doesn't like to share file system partitions.  Chrome sandboxes each site to prevent sharing.  This is pretty nice.  If my Youtube tab has a problem, I don't have to consider debugging the Reddit tab.  If I upgrade one project's Node dependency, it doesn't break other Node projects.

Some cultures that used to favor sharing are moving away from it.  Perl (local::lib) and Go (vendor directories) come to mind.  Of course, everyone can still share (in the sense of using identical code), they just get their own local copy of it.  Go's default of building static binaries is a breath of fresh air.

All of this is just a reminder to me: if my code shares anything, find a way to make it either local (not shared) or immutable.

Thursday, March 02, 2017

Caller ID for SMS in XMPP

When you receive an SMS on a traditional mobile device, your phone looks up the number in your local address book and displays the corresponding name.  That works great for a small social network where you can maintain the list yourself.  As the number of potential contacts (coworkers, telemarketers, etc) increases, it becomes unwieldy.

For example, I routinely receive SMS from people in my ward or stake who I'm unlikely to contact on a regular basis.  That's a few thousand people and the population changes weekly.  I don't want all that info in my regular contact list.

Fortunately, I run my own little phone system that uses XMPP for sending and receiving SMS.  I wrote a gateway to handle the conversion back and forth between those networks.  That gateway can run a script to perform caller ID however I want.  So now I have something like this when an SMS arrives:

  1. check my personal address book
  2. check the stake directory
  3. check 800notes for records of telemarketing
I also have multiple external phone numbers.  One of them is a junk number that I give out when I fill out online forms.  If an SMS comes in to that number, the caller ID appears with a "(junk)" prefix.

It's a lot of fun to be able to write a couple shell scripts to customize how your phone works.

Wednesday, March 01, 2017

Signing up for Project Fi

In preparation for an extended visit to Europe, I signed up for Project Fi today.  Prior to this I had been using Ting.  I love Ting.  I do all my SMS and voice communications over IP, so I really just need a reliable, affordable data plan.  If Ting had better data rates in Europe, I would have stuck with them.  Anyway, here are some first impressions while signing up for Project Fi.

Charging $20 per month per phone line seems expensive.  That costs $6 on Ting.  Fi gives you unlimited calls and texts but since I always use zero of those, I'd rather have a cheaper base rate.

Fi charges $10 per GB for data.  As they say, "No matter your budget, you'll only pay for data you use."  So why are they asking me to set a budget?  It seems like an extra step without a purpose.  I chose the smallest budget allowed, 1 GB.  This combination costs $30 per month on Fi and $22 on Ting.  An extra $8 per month to get excellent data rates across Europe seems reasonable to me.

I already have a Google Pixel so I only needed a new SIM.  That was free.  As I recall, Ting charges $12 up front for that, so $0 is nice.

Since my public phone numbers are all managed through Twilio, it doesn't really matter to me what phone number Fi assigns me.  They gave me the choice of porting a number or choosing a new one.  I chose to receive a new one.  Unfortunately, they only offered numbers with area codes in my local area.  Some people might like to choose an arbitrary area code.  They didn't let me choose the specific number at this point.

Fi asked for my primary service address.  Since I'll be using this phone all over Europe, I wasn't entirely sure what to put.  I used my home address in Wyoming.  Fi didn't seem picky about that.  They even let me use a different shipping address for the SIM card.  That was convenient since I was traveling at the time I placed the order.

Overall the sign up process was smooth and the prices seem reasonable.  When the SIM arrives in 7 days, I'll put it through its paces.

Tuesday, February 07, 2017

Selfhosted Google Voice

I've used Google Voice since 2010.  During that time, it's been my main telephone number.  Besides Search, it's my favorite Google product.  Google has even renewed their commitment to the product recently.  Unfortunately, Google Voice only works for US customers and my family and I are moving to Europe later this year.  GV has spoiled me and I don't want to return to the old days when I used my mobile carrier's number directly.  This is how I'm selfhosting an alternative to Google Voice.

Twilio

Today I finished porting my Google Voice number to Twilio.  They offer a great platform on which to build telephony apps.  I've been very impressed with their customer support too.  It's refreshing to be able to discuss telephony problems with your provider.  That was never possible with Google Voice.

They offer phone numbers in every country we plan to live.  They provide numerous telephony services, but we're only using a few right now.

SMS

I selfhost an XMPP server for my family.  We use it all the time for messaging each other.  We run Conversations on mobile and use other XMPP clients on desktop.  It works great.  I wrote a gateway between XMPP and SMS.  You send an XMPP message and it converts it into an SMS.  When someone sends me an SMS, it converts it into an XMPP message.

This all works through Twilio's SMS API.  Since the whole system is under my control, I can add arbitrary logic for handling SMS.

Voice

To replace the voice component of Google Voice, I set up an Asterisk instance.  I use Twilio SIP trunking to connect to the telephone network.  I run Zoiper on my Android phone.  To make an outgoing call, I use the Zoiper dialer and Asterisk routes the SIP call through Twilio to the public telephone network.  When someone calls my Twilio number, Asterisk routes the call to Zoiper on my Android device.  The call quality is great.

I have no previous experience with PBX systems, but it was all surprisingly easy to configure.  Once again, since it's all in my control, I can add whatever features I want.

Future

For now I'm using a very basic setup.  SMS accounts for 90% of my Google Voice usage and that part works really well.  Even if I never had more than this, I think I'd be content.  I have a bunch of features in mind and this setup makes it really easy to add them as I need them.