Hello! I’ve been working on writing a zine about debugging for a while (here’s an early draft of the table of contents).

As part of that I thought it might be fun to read some academic papers about debugging, and last week Greg Wilson sent me some papers about academic research into debugging.

One of those papers (Towards a framework for teaching debugging [paywalled]) had a categorization I really liked of the different kinds of knowledge/skills we need to debug effectively. It comes from another more general paper on troubleshooting: Learning to Troubleshoot: A New Theory-Based Design Architecture.

I thought the categorization was a very useful structure for thinking about how to get better at debugging, so I’ve reframed the five categories in the paper into actions you can take to get better at debugging.

Here they are:

1. learn the codebase

To debug some code, you need to understand the codebase you’re working with. This seems kind of obvious (of course you can’t debug code without understanding how it works!).

This kind of learning happens pretty naturally over time, and actually debugging is also one of the best ways to learn how a new codebase works – seeing how something breaks helps you learn a lot about how it works.

The paper calls this “System Knowledge”.

2. learn the system

The paper mentions that you need to understand the programming language, but I think there’s more to it than that – to fix bugs, often you need to learn a lot about the broader environment than just the language.

For example, if you’re a backend web developer, some “system” knowledge you might need includes:

• how HTTP caching works
• CORS
• how database transactions work

I find that I often have to be a bit more intentional about learning systemic things like this – I need to actually take the time to look them up and read about them.

The paper calls this “Domain Knowledge”.

3. learn your tools

There are lots of debugging tools out there, for example:

• debuggers (gdb etc)
• browser developer tools
• profilers
• strace / ltrace
• tcpdump / wireshark
• core dumps
• and even basic things like error messages (how do you read them properly)

I’ve written a lot about debugging tools on this blog, and definitely learning these tools has made a huge difference to me.

The paper calls this “Procedural Knowledge”.

4. learn strategies

This is the fuzziest category, we all have a lot of strategies and heuristics we pick up along the way for how to debug efficiently. For example:

• writing a unit test
• writing a tiny standalone program to reproduce the bug
• finding a working version of the code and seeing what changed
• printing out a million things
• adding extra logging
• taking a break
• explaining the bug to a friend and then figuring out what’s wrong halfway through
• looking through the github issues to see if anything matches

I’ve been thinking a lot about this category while writing the zine, but I want to keep this post short so I won’t say more about it here.

The paper calls this “Strategic Knowledge”.

5. get experience

The last category is “experience”. The paper has a really funny comment about this:

Their findings did not show a significant difference in the strategies employed by the novices and experts. Experts simply formed more correct hypotheses and were more efficient at finding the fault. The authors suspect that this result is due to the difference in the programming experience between novices and experts.

This really resonated with me – I’ve had SO MANY bugs that were really frustrating and difficult the first time I ran into them, and very straightforward the fifth or tenth or 20th time.

This also feels like one of the most straightforward categories of knowledge to acquire to me – all you need to do is investigate a million bugs, which is our whole life as programmers anyway :). It takes a long time but I feel like it happens pretty naturally.

The paper calls this “Experiential Knowledge”.

that’s all!

I’m going to keep this post short, I just really liked this categorization and wanted to share it.

[cw: violence] I just wanted to capture this story here once so I don't have to tell it to anyone again. Because, while I am okay now, it is unpleasant to keep having to repeat the story to new people over and over. The short version is, I was violently assaulted by a man who was evidently going through a significant mental health episode, I am physically okay even though the experience was fairly traumatic in the moment, and we are going to try to get the man help if it's possible.

I was attending a professional conference in a small town in California that I had never been to before. I went for a walk around the area we were staying, since I always like to see a little bit about a new place I’m staying at. About a half mile from the hotel I was staying in, there was a small coffee shop on the main road, so I went in to get a drink and decided to sit at one of the picnic tables out front of the shop.

There had been a disheveled man sitting at another of the picnic tables when I had entered the coffee shop, but I hadnt really paid any attention. As I sat down, I noticed he had a cart with a lot of his stuff, so I figured he must be unhoused, but didn't otherwise have any thoughts about him. I set my drink down and got out my phone as usual when he almost immediately started walking over. I began to reach for my wallet, assuming he was going to ask me for a couple bucks, and he put his hand on my shoulder in a pretty heavy way, more than you would do to get someone's attention.

He started to say, "I'm sorry man…" and I assumed he was apologizing for being about to ask for money, but then he finished his sentence, saying "I'm sorry man, but I'm going to have to kill you." and sucker punched me in the face just near my left eye, then started screaming and swinging at my head.

My heart was racing immediately and I said “I think you’re mistaking me for someone else” or something similarly stilted (which in retrospect is pretty absurd) and I started to back away from him and tried to get up from the table when he tried to lunge at me where I was sitting.

I stood up from the bench and pushed him away (kind of a big shove) and said “I’m not trying to hurt you, leave me alone” and then fully lunged at me and fell on top of me and started punching and screaming. He was saying, “I am god and you are the devil, and you are going to die today!” and stuff like that. I was shoving him off and trying to grab his wrists because I had seen some kind of glint of metal and I was worried he had a weapon; I think in retrospect it may have been a carabiner or something like that clipped to his clothes. It was probably a full minute later until I knew for sure he didn't have a weapon in his hand, so this was the scariest part for me. (A minute is quite a long time in this kind of circumstance.)

We tussled on the ground and I kept trying to get control of his hands and I was yelling "Help! I don't know this guy!" because I have some vague recollection that you’re supposed to say that as it’s effective in engaging bystanders. That idea may be apocryphal, but it did work, as a man who had been sitting by the coffee shop came over and started yelling for the man to get off of me. I was also screaming "Help me!" because I was still pretty terrified at that point. I did not want to die or be seriously injured because I was too reticent to ask for help.

Once we got to the ground, the attacker was sort of slumped over me, and he probably outweighed me by 50-75 pounds, so when I would kick him off of me, he would sort of just drop onto me again, and I realized if I fully got up, I would have to disable or restrain him to stop him doing that. So instead, I just stayed focused on keeping his hands pinned to the ground. This had the unfortunate and disgusting side effect of being smushed up directly against someone who was extremely dirty and smelled terrible, but it significantly lowered my sense of immediate danger.

At that point, I started trying to de-escalate, as we were in something of a steady state and I could now see he had no weapon. He began sort of angrily looping through a couple of different angry ideas, screaming variations of refrains like "a girl put a knife in my face", "my money is no good here" and "you're Al Qaeda, you did 9/11" amidst general shouts of how I was evil. None of it was obviously very coherent, and I didn't catch it all because I had been also yelling a lot, but I just kept saying stuff like "I don't know you, man, and you don't know me" and "I've got no problem with you, we don't have to do this" and reminding him I'm just some guy from New York who was visiting and I have a wife and a kid back home. At one point after he said the stuff about 9/11 again, I said, "No man, I'm a New Yorker! I was there that day!" which really backfired in an absurd way because he heard it as me saying I was there for 9/11. The next time I am assaulted by a stranger, I will hire an editor.

In between his outbursts, the man had seemingly genuine looks of confusion and disarray and disorientation. Obviously, I am not at all qualified to make any medical diagnosis, but having seen people go through mental breaks, his behavior seemed consistent with that sort of episode. I figured this was probably not the first time he'd been through this, so I said "Let me up, man, you don't want to catch a case on this" and he responded that since I was evil, the cops would be taking me away. This gave me the idea to then to tell him "okay, let's just wait and do that" since I figured someone had already called the cops by that point, and so things calmed down a bit even though we were still lying in a sort of disgusting pile in the parking lot.

The nice helpful man who had come over (I later found out his name is Michael) was in my line of sight and I mouthed a thank you to him for staying with me and he said confirmed that the cops would be there soon, while he also kept exhorting the attacker to calm down and leave me alone. After a while, the guy was sort of just muttering at me while I still had his arms pinned, and one point when I said I just wanted him to back off because I'm trying to get home to my wife and son, he said "I got a wife too, had three of 'em" and I said "I can see why they don't stick around" and he actually laughed.  After that I focused more on telling him versions of, "this isn't you, man, we're gonna get you some help" and he scoffed at that but did let up a bit on struggling agains me trying to subdue him. At one point I even let go of one of the attacker's arms to take Michael's hand and say thank you again, because while I was still basically lying on the ground, I could see now that I was gonna be okay and I got a little emotional about it.

Afterward

The cops came after what was probably only a few minutes (though it felt like much longer) and pulled the guy off of me, and since they couldn’t really see me or know what was up, I just kept repeating “I’m staying down, I’m not getting up, I don’t have anything, I don’t know him” with my palms up and out. The cops grabbed me and frisked me while picking me up, but Michael kept saying “he did nothing and was attacked” and overall the cops were not violent or aggressive. They sat the attacker guy down on the ground and I sat a good distance away near the coffee shop. I caught my breath and the young barista very sweetly brought me a new drink. I absolutely reeked, and as I looked around on the ground I found my phone undamaged but my sunglasses were smashed, my shirt that I was wearing (one of my favorites!) was totally ruined with rips and dirt, and my fresh pair of white sneakers were absolutely filthy. Obviously none of that actually matters, but for whatever reason that's what bothered me in the moment, and what I noticed most as I gathered myself. I felt a little sore or bruised near my eye but otherwise felt physically fine. I figured I must have looked a lot worse than I felt because people were reacting to me like I was hurt.

The guy who attacked me was getting around okay as far as I could see, and I had only really kicked him enough to give myself space once we were on the ground, and I hadn’t hit him at all, so I don’t think he was hurt physically. The cops tending to him were fairly respectful and did not manhandle him at all. Based on how he had phrased some things (and frankly, due to his being unhoused and severely mentally ill), I think the man who attacked me was likely a retired veteran, which may have helped with the cops.

The cops started asking me questions, basic stuff like if I knew the man or what he and I had said to each other before the altercation, and then asked if I needed a medic. I said that I did not but paramedics soon came anyway and first they checked on the other guy, then checked on me and my vitals, and did basic tests to see if I had a concussion or anything. I just kept telling them I was fine so they finally stopped after I signed their iPad thing confirming that. The cops asked me four or five times if I wanted to press charges and I kept saying, “just help him, this man isn’t well, I don't need anything” and two younger cops at least seemed like they would try to do so. The third older cop I said it to was kind of rolling his eyes, and I think part of the reason I kept getting asked the same questions over and over was because at least one of them was in training. When they asked if I wanted to report this as a hate crime (I did not), they all asked in exactly the same phrasing, which suggests to me that they've probably all been trained on this relatively recently. When I kept repeating, "I'm going home to my family tomorrow, that guy isn't", they finally eased up on pushing me on that stuff.

Overall, about 6 cops in total showed up, along with 2 paramedics. Things calmed down with the staff and patrons of the coffee shop and nearby store, so by this time I was just sitting in front of the place catching my breath, and that's when I asked Michael his name. He responded and also offered a lot of very thoughtful forms of material support, everything from taking me his house to take a shower if I needed, to giving me a ride back to my hotel. I told him I had just gotten a gift of being reminded that there was kindness and courage all around us and I was grateful for him. We made small talk about how he was sorry I came all the way from NYC and had this experience. Later that evening, he emailed me and said he was going to a nearby K-pop festival with his family; I stopped by but we missed each other.

I went into a little clothing shop that was adjacent to the coffee shop to buy a new, clean shirt, and then used their bathroom to wash up. That was when I first got a good look at myself. I saw my eye had some broken blood vessels and I had blood on my face, especially near my chin, though I still don't know where that came from as I never found any cuts there. In general, I think I must have looked like I’d lost a fight. (Which is not how I had felt.) But my appearance in the mirror was jarring, so I washed my hands and face as best I could; naturally, there weren't any paper towels to dry off with.

When I came out, I saw an industry colleague who had been at the conference I had been attending. He had coincidentally decided to walk to the same coffee shop and had apparently arrived after the cops had already settled everything down. I asked him if he’d walk back to the hotel with me and he was super gracious about it. After getting his contact info and thanking him again, I told Michael that he was off the hook and he could go home since I had run into a friend who would walk me back.

I saw the cops loading the other guy into their car so I asked them if I could get his name as I wanted to see if I could help make sure he would get some support. They said they couldn’t give me his info since he was now in medical care and that would be a privacy violation, but they did take my name and number so the medical staff could offer it later. They also said I could contact them if my injuries ended up being worse than we thought or if I changed my mind about pressing charges.

As we walked back, I started to process the experience, noticed other cuts and scrapes and bruises that I'd missed, and caught more weird absurdities about what had happened. (My Apple Watch was giving me an alert that I had "taken a hard fall", a feature I had forgotten existed.) After a shower and some rest and talking to family, I pretty much felt like myself again. Notably, a number of friends and industry acquaintances, largely men with whom I'd only shared a professional relationship, offered me hugs and surprisingly sincere support. It's a minor takeaway, but seeing emotional vulnerability being so easily accessible and immediate did make me feel like that perhaps business environments don't have to be as heartless as they are by default.

Takeaways

I don't, at least as of yet, have any great social lessons or grand pronouncements from this experience, except for the obvious ones. We are failing so many in society by not having a comprehensive, effective social safety net. The town I was in had just increased funding to their police department a few years ago for a "homeless outreach team", yet when I asked them how they would be helping this person afterward, there was no coherent or specific plan. If, as I suspect given that the area has the largest military population in the country, the man who attacked me is a veteran, we've increased military funding to record levels while failing our veterans more profoundly than ever. The decades of repeating destructive, demonizing narratives about vulnerable communities have a huge effect on people in crisis, becoming part of how an unhealthy mind lashes out at the world.

Our institutions have no capability for responding to crisis with compassion.

The municipality where this took place had just closed a facility a few weeks ago that had been serving unhoused people just a few blocks away, and the new facility that is supposed to take its place hasn't opened yet, and won't open for months at least. (I made a donation in Michael's name to the organization that will be running the new facility; if you'd like to do the same, please let me know and I'll match your gift.) I am wealthy and privileged and have physical and mental health care, along with the resilience granted by having the support of family and friends and community who care, so I'll get past this relatively quickly. But someone else in this position, or even if this had happened to me earlier in life when I was less privileged and more vulnerable, could very easily be permanently thrown off track by trying to recover. I don't expect that the man who attacked me will ever have a stable life or be fully supported, and he deserves to be helped and healed, and I don't yet know what to do about that except try to support the institutions which are trying.

In short, I’m okay, but I’m really really sad. Everything about this makes me so sad, for reasons best encapsulated in what Michael later emailed me: “Our country faces so many problems—a lot of that seemed to come out at the cafe unfortunately” which I think is an eloquent and unfortunately apt summation. But I also saw a stranger show  love, compassion, courage and kindness in a moment of genuine fear, and I am hoping that's the part of this experience that endures.