Jerry’s Story: An aspiring auto mechanic changes course

Part 1 of my telling of Jerry Weinberg’s story was Jerry’s Story: First interactions, where he had started his college studies in September of 1950. But let’s go back to the summer of 1950, when Jerry had no plans to attend college at all.

Jerry had graduated from Omaha Central High School, and he felt disgusted with school. He found many of the subjects in high school to be trivial, so he had skipped most of those classes and still got good grades. He did enjoy a few classes, however, especially auto shop. He told me “I just loved cars, driving them, working on them, even washing them—plus doing body work and painting in my father’s shop. I never really had any other career idea than working with cars in some way.” Though he was fascinated with computers, there were so few jobs available to work with them at the time (and none that he was aware of) that he didn’t even consider a computer job a viable option.

After graduating from Central High, Jerry applied for a job as a mechanic. The owner of the garage offering the mechanic job, however, wouldn’t let him start until after the next school term started. He suspected that Jerry was just looking for a summer job, but he really wanted a long-term employee. Jerry decided to wait out the summer so he could get that job, and in the interim, he worked as a summer camp counselor for a camp sponsored by the Omaha Jewish Community Center. At the camp, another counselor encouraged him to go to college so he could meet young women. Jerry had a keen interest in women, and hadn’t before considered this particular benefit of the college experience. So he determined to go to college instead of taking the mechanic job.

A few days before classes started in the Fall, Jerry showed up at the University of Nebraska in Lincoln to register. The counselors were not happy that he hadn’t registered in advance, but because he had graduated from a Nebraska high school, state law required that they admit him. The counselors were even less happy to find that they had to give him a scholarship because he had graduated in the top ten percent of his class. So he began his studies.

While at the university, Jerry got a job in the Physics department—the job title was “computer.” It turned out that Jerry would be a computer years before he programmed one. He used a Friden electromechanical calculator along with pencil, paper, and eraser to invert 10 by 10 matrices for faculty members. Just as a computing device doesn’t know the ultimate reason it does its work, he doesn’t recall ever knowing why they wanted the inverted matrices. Jerry told me about what he learned from this job:

I recall that it took me upwards of an hour to invert a 10 × 10, and of course the inversion time tends to grow as the square of the size. Going to 11 × 11 would have raised my computation time by over 20%, and increased my chance of making an error somewhere along the line. That was the first time I became aware of non-linear computation times and also the significance of error. It was a good start to my career: my understanding of these factors, which many programmers today don’t seem to appreciate.

He also offered his services as a tutor for any subject, primarily for failing athletes, and he worked grading English papers. He was a Physics teaching assistant and was told he was the first undergrad to get that job, at the ripe old age of 17.

Jerry was out sick with Crohn’s disease for most of his second year. He went home to Omaha to recover. While there, he took a few courses at the University of Omaha (now known as the University of Nebraska Omaha), including Mathematics of Finance. He thought that computers would be used in course, but he had no such luck. There was most likely no computer on campus at all.

When the course progressed to more advanced subjects like probability, statistics, and risk, Jerry found out he knew more about them that the professor did, so he helped teach the class. This impressed the instructor, who was an associate of Warren Buffet. The professor recommended that Jerry meet Buffet because he was seeking bright math students to work with. Jerry wasn’t able to arrange a meeting, however, because he had to return to the hospital for surgery.

During his stay in Omaha, Jerry did manage to meet with the chief actuary at Mutual of Omaha. Jerry was impressed with the luxurious office, but not impressed with the actuary job itself.

He returned to the University of Nebraska and completed his Bachelor of Science degree, magna cum laude and with honors, for four majors: Physics, Math, Philosophy, and English. He then moved to California to study Physics at the University of California, Berkeley. A year later, he had passed his comprehensive exam, finished his thesis experiments, and was on track to earn a PhD in record time. He had a few months of work left to finish writing up his thesis when he found the opportunity he had been looking for since he was 11 years old.

Jerry was out of cash, and supporting a wife and a child, with the second child on the way. He read an ad in Physics Today from IBM looking for applied science representatives. It didn’t say that the job involved computers, though there was a picture of a roomful of data processing equipment. It’s not the first computer-related job ad that Physics Today ran, but it was the first that he noticed. He had no doubt that this was what he wanted to pursue. He wrote to IBM to apply, interviewed in Oakland, and was offered a job on the spot. He also interviewed at Boeing and got an offer for more than twice what IBM offered, but the job did not involve computers.

Accepting either job would mean not finishing his PhD. Jerry says “The degrees were irrelevant to me, but came along as a side effect of my hanging around. My advisor actually cried when I told him I was leaving.” He received a Master’s degree in Physics from UC Berkeley as a consolation prize. Jerry was hired for his dream job as an applied science representative at IBM on June 1, 1956.

I’m sure that Jerry would have found a way to play with computers before long, even if it weren’t for that wary garage owner, the fortuitous advice from his fellow camp counselor, or the worry about paying his family’s expenses. But I was fascinated to see the path that he took to realize his dream.

An excerpt from the ad in Physics Today, which ran in the January 1956 and March 1956 issues. You can see a full scan of a very similar ad from the February 1956 issue of Scientific American.


Can you help to provide additional details from your own knowledge of this era or from your interactions with Jerry? Please comment here or contact me on Twitter.

The next installment in this series is Jerry’s Story: Jerry, the Real Programmer.


A bit of advocacy helps to earn a bug bounty

I have been working on honing my security testing skills. I asked Don Ankney‘s advice on how to do this, and one of his suggestions was to participate in bug bounty programs. Many companies encourage security researchers to report security vulnerabilities to them, and in some cases, they offer monetary rewards to the first person who reports each one.

My first bug bounty report for Instagram, which wasn’t accepted, was discussed here: “Username Enumeration – Ho, Hum!” This time, though, I was more successful. I found that none of Instagram’s cookies on its web interface had the “secure” flag set, including the session cookie that identifies a logged-in user. Like username enumeration, the secure flag on the cookies is another “ho, hum” thing often excluded from bug bounty programs. But the Facebook Bug Bounty Program (which also covers Instagram) doesn’t mention such an exclusion, so I decided to report the vulnerability.

I spent some time crafting an attack scenario. I found that the attack didn’t work if I used “” instead of “” I found that if the insecure page was in the browser cache, the browser used the cached page and then there was no vulnerability. And for reasons I haven’t figured out, I was not able to complete the attack successfully if the victim was using Firefox. I was able to prove that hijacking an Instagram session was a simple matter of setting just the captured sessionid cookie. This is the bug report I sent:

Description and Impact

The secure flag is not set on any of Instagram’s cookies, including sessionid. When a user with an active session types “” in their browser to go to the site, they will first hit the insecure site and transmit all of their cookies in the clear. An attacker monitoring their network packets will be able to hijack their session easily. Assuming there is no need to send cookies in the clear at any point, this is easily fixed by setting the secure flag in the cookies.

Reproduction Instructions / Proof of Concept

I implemented a proof of concept using Safari 8.0.8 on Mac OS 10.10.5 and Chrome 49 on Windows Vista Home Basic for the victim. I haven’t been able to reproduce it yet with Firefox.

  1. Make sure you’re not logged in to Instagram. Clear the browser cache.
  2. Go to
  3. Click “Log in with Facebook”, and enter valid Facebook login credentials. This logs you in to Instagram.
  4. …an arbitrary amount of time may pass, as long as the Instagram session is still valid when continuing.
  5. Go to a public network that someone is snooping on.
  6. Open a tab in the same browser as before and go to (not https). The sessionid cookie is sent in the clear and has been captured by the attacker. Even though the server returns a 301 redirect to a secure site, the cookie has already been sent in the clear.
  7. Attacker hijacks the Instagram session by setting the sessionid cookie in their browser.

I got a reply five days later, saying “This is currently intentional behavior in our product…” I wasn’t surprised that another “ho, hum” bug was rejected, but I was surprised that they considered it a feature. So I replied, saying that I intended to publicly disclose the issue (which is standard practice after the report is closed, whether fixed or not) and I asked for further information about how the site needs this behavior in order to function, to inform my continued testing. I call this sort of response my “Just one more thing” reply, inspired by the TV character Columbo. This sort of followup is routine for professional software testers, but I don’t know how many security penetration testers put bug advocacy skills to use.

The next reply came quickly, saying that though many people had already reported this issue, they would go ahead and discuss the issue with the product team and try to fix it. And lo and behold, about three weeks later, I got notice that the issue is resolved, and I was pleasantly surprised to hear that they offered to pay me a bug bounty. The reasoning was fascinating – the site previously used http (I’m not clear how long ago) and then later switched to https. All the previous reports about this issue had been when they used http, which is silly, since in that case the secure flag would render the cookies invisible to the server. This explains their earlier pat rejection of bug reports about the secure flag, even though that response had become obsolete with the change to https.

They determined that I was the first to report the vulnerability since they switched to https, and so I qualified for the bounty. I am impressed with the amount of care that Facebook/Instagram took in handling this report. I’m eager now to dig deeper and apply more of my bug advocacy skills if necessary.



Jerry’s Story: First interactions

My friend Jerry Weinberg was present at the dawn of the age of computers. He can describe first-hand what that was like, but much of his story has never been told. I have started to collect Jerry’s stories. Here is a small sampling, which I prompted by asking him “What were your first interactions with a computing device?”

The first thing that Jerry used that we could call a computing device is a slide rule that his father, Harry, gave him when he was about 7 years old. Harry worked for more than 20 years helping to improve processes at Sears, Roebuck & Co. He bought slide rules in quantity to give to the young ladies who computed customer bills. They used slide rules to check their multiplication, for example, when multiplying price times quantity. This practice caught an enormous number of errors before the bills were sent to customers.

Jerry had a more interesting use for his slide rule, though. He was a sports fan, so he used the slide rule to compute baseball batting averages. Jerry says “It’s the easiest thing in the world. A 7-year old could do it.” He still has that slide rule.

Jerry slide rule

He has a “UNIQUE” brand slide rule, made in England. Jerry describes it as small and cheap, with a table of “Trigonometric Ratios” on the back, which he didn’t understand how to use when he started using the slide rule. Later, though, he remembers using tables of sines, cosines, and logarithms, which could also be considered computing devices of a sort. He used the tables in math classes and also for experimenting with numbers for fun.

Jerry remembers his first introduction to the concept of computers being a Time magazine article. This may have been “Science: A Machine that Thinks,” in the July 23, 1945 issue, when he was 11 years old. That article discusses Dr. Vannevar Bush’s “memex,” a conceptual idea of a machine that stores facts for easy recall, which the Time article refers to as a “brain robot.”

The first book that Jerry read about computers was Giant Brains, or Machines That Think, by Edmund C. Berkeley, published in 1949. This book had a strong influence on Jerry, and he considers Berkeley one of his heroes. Much later, he met Berkeley and had long conversations with him, and he was delighted to know that Jerry had been inspired by his book.

Jerry may also have seen “Science: The Thinking Machine,” the landmark Time cover story on January 23, 1950, when he was 16 years old. The cover artist for that issue, as it was for many issues of Time, was Boris Artzybasheff, which is a detail that Jerry still recalls. The article discussed the Harvard Mark I named “Bessie” (which coincidentally is also Jerry’s mother’s name). This was an electromechanical computer that had been in operation since 1944. The article also discussed the Harvard Mark III, a hybrid electronic/electromechanical computer produced in 1950 and it went into detail comparing computers to the human brain.

Jerry was an avid reader. He explained just how avid: “I usually had breakfast alone, with cereal, so there was the box to read. I’m not saying it was my preferred reading, but just that I read everything that appeared in front of me. Like the see-food diet: I see food, I eat it. So, I see print, I read it.” He probably heard about computers from other sources during his youth. He remembers sitting at his father’s feet as his father read the newspaper and offered his commentary on a wide array of topics.

Jerry had been labeled as a “brainy” kid, and he yearned to learn more about brains, especially these “giant brains.” Early on knew he wanted his life’s work to be with computers. He didn’t yet know anyone who had ever seen a computer, let alone used one. He watched and waited for signs of a computer, but went all through high school without seeing one, with perhaps one exception. He had a summer night job in a large bakery computing recipe requirements for the following day’s orders. He used a Monroe adding machine.

When he entered college at age 16, Jerry told his counselors that he wanted to work with computers, but none of them knew anything about computers except that they had something to do with electrical engineering and physics. They decided he should major in physics because he was good at math, which they thought would be wasted in electrical engineering.

One day, Jerry saw a notice for a brief “computing course” using Monroe adding machines, given by the Monroe company. He already knew most of the material better than the instructor. He passed, earning a certificate that he’s lost somewhere along the way. It’s the only computing course he ever took, and the only “degree” in computing that he ever earned.

If you’re interested in hearing more of Jerry’s story, please let me know. He has much to tell. Note that many of the words above are his, and I decided to tell the story in third-person. Consider it a collaboration.

The next installment in this series is Jerry’s Story: An aspiring auto mechanic changes course.

Username Enumeration – Ho, Hum!

I used to think that checking for username enumeration vulnerabilities was important to do. Based on what I’ve observed, now I’m not so sure.

When I’m conducting a broad test of a software system, I tend to check for basic security holes like username enumeration. “Username enumeration” vulnerabilities happen when software that has login accounts for its users gives you a way to build a list of valid login accounts, for example, by giving you a way to query whether a guessed username is valid or not. Once you have this list, you can try to launch a brute force attack to guess the passwords.

Testing for username enumeration is one of the mitigations recommended in the 2013 OWASP Top 10. Cigital gives it even more prominence in their Top Web Application Security Vulnerabilities Compared to the OWASP Top 10, with username enumeration ranking as the 9th most commonly found, even when only considering the vulnerability via password reset features. My experience, though, is that companies aren’t very interested in fixing these vulnerabilities. It may be commonly found because it isn’t commonly fixed.

Once I did get a fix for an obvious vulnerability, but when I found I could still do enumeration by checking the response time from the server, that didn’t get fixed. I often see no fix at all when I report a username enumeration problem. For example, the main web login form for Instagram has this vulnerability, but Instagram considers your username to be public information, and they confirmed when I contacted them that they’re not going to close this hole. A significant fraction of the companies that participate in the HackerOne bug bounty program specifically state that they exclude username enumeration from the program.

I’ve found a few ways that companies have indirectly mitigated this issue, which may be contributing to some of the “ho-hum” response:

  1. Rate-limiting on the vulnerable feature based on IP address. In my testing, it wasn’t uncommon to see that a feature that allowed me to enumerate would temporarily lock me out after about several tries in rapid succession. This would only succeed in locking me out if it’s based on my network presence, not on the username, since I would be trying a different username each time.
  2. Similarly, a few vulnerable sites use CAPTCHA to defeat automated enumeration attempts. After trying several different usernames, I would get a CAPTCHA challenge that stopped a script from continuing.

In both of these cases, I can easily determine if a particular username is in use. But if I want to compile a large number of usernames, it may take a script months of running at the maximum allowed rate. There may be other ways to defeat these measures, such as frequently changing the public-facing IP address, using a botnet, or trying to use a database of known CAPTCHA responses.

None of this matters for Instagram, because the vulnerability on the web login is not rate-limited in any way. And that isn’t even the easiest vector – the issue with incremental userIDs mentioned here has not been fixed: InstaBrute: Two Ways to Brute-force Instagram Account Credentials.

One remaining mitigation comes to mind – the account lockout based on failed password attempts. Once we have a list of good accounts, we haven’t gained anything until we guess the password. I haven’t tried cracking that nut yet. My first thought is that if we have a large number of usernames, the time it takes to try one particular password for each of them may not trigger an account lockout at all by the time we roll back to the top of the list for the next password to try, as long as the lockout feature automatically resets itself after some fairly short period of time. But perhaps I’m being naïve about how quickly we would need to progress through a long list of possible passwords.

I would be curious to hear what your experience has been with reporting username enumeration problems, especially with companies that set a high priority on closing these holes.

Image credit: Christiaan Colen (CC BY-SA 2.0)


Better Bisecting


Bisection is a procedure that software testers can use to isolate a defect. I’ve been having fun building a tool to assist with bisection, and I’m writing about it in order to get feedback on whether it may be useful.

This work was inspired by the “bisect up” and “bisect down” features that James Bach developed for the perlclip tool. These features are tied to the counterstring generator. You start by creating counterstrings of two different sizes (perhaps vastly different sizes), where generally you see that when you use the smaller of the two strings in a test, it passes, and you observe that a test with the larger string fails. The task is then to determine precisely where the boundary is between passing and failing. You can use the “u” (bisect up) and “d” (bisect down) commands depending on whether the last test passed or failed to generate additional test strings that bisect the remaining possibilities until you converge upon the boundary.

I’ve used this feature many times – it’s really helpful. But I often find that I get confused trying to keep track of whether I should go up or down. If I make a mistake, I have to back to the last two values I’m confident in, create two counterstrings that I don’t actually use, and start over bisecting from there.

Here is my redesign that I think makes bisection easier to do. This is implemented in testclip, my work-in-progress port/enhancement of perlclip using Ruby. I’m going to demonstrate the tool by finding a bug in Audacity (version 2.1.2 on Mac OS).

I created a new Audacity project and looked for a text field that would be a good candidate for long string testing. I clicked File > Edit Metadata and found what I needed. I fired up testclip and make a counterstring for a happy path test:

$ ./testclip.rb 
Ready to generate. Type "help" for help.

cs 10
counterstring 10 characters long loaded on the clipboard

Then I pasted the data into the first field on the Metadata Tags dialog in Audacity:


I clicked “OK,” then opened the dialog again, and the data was preserved without any errors. So I established that the counterstring format is valid for this field (some input fields don’t allow asterisks or numbers, so it’s good to check). I recorded the result of the test in testclip:

10 recorded as pass

That was boring. Next I wanted to try a large number. In my experience, 10,000 characters is very fast to generate, and larger than what most input fields need, so that usually where I start. I asked testclip for a 10,000 character string.

cs 10000
counterstring 10000 characters long loaded on the clipboard

This was the result of the test:


When I tried to move the cursor further to the right, the cursor moved off the window, and I couldn’t see any more of the string. Looks like I found a bug. I record the result in testclip, choosing the tag “obscured” to identify the particular failure. This may become important later if I find a different failure mode.

fail obscured
10000 recorded as fail obscured

Due to the nature of the counterstring, I already had it pretty well isolated – it’s likely that my boundary is between 5690 and 5691 characters. But let’s make sure. I generated a 5690-character counterstring, find that it works fine, and recorded that as a pass.

cs 5690
counterstring 5690 characters long loaded on the clipboard

5690 recorded as pass

I can ask testclip to report the test results it knows about and automatically identify where the boundaries between different results are.

10: pass
5690: pass
--boundary 1
10000: fail obscured

Next I tried 5691. This failed as with the 10,000 character string. I recorded this as the same type of failure and show the status again, which shows that testclip puts 5691 and 10,000 in the same equivalence class, just as it did with 10 and 5690 (I’m hoping it’s not confusing to call this an “equivalence class”, which is a term usually used to describe expected results, not actual results).

cs 5691
counterstring 5691 characters long loaded on the clipboard

fail obscured
5691 recorded as fail obscured

10: pass
5690: pass
--boundary 1
5691: fail obscured
10000: fail obscured

So, I hadn’t needed to bisect anything yet. I decided to make the test more interesting by saving the project and opening it again, to see if the long string is saved and loaded properly. I went back to 5690 and did the save and load test. Note that generating a new counterstring would reset the bisection status in perlclip, but all the test results are still retained in testclip so I can track multiple failure points. And in fact, the test fails, because after I open the saved file, the field is completely empty.

So now I abuse the tool just a bit, changing the result of the 5690 test. I’m actually running a slightly different test now, but I think I can keep it all straight. I tag this new failure “empty”. I now have two boundaries:

cs 5690
counterstring 5690 characters long loaded on the clipboard

fail empty
5690 result changed from pass to fail empty

10: pass
--boundary 1
5690: fail empty
--boundary 2
5691: fail obscured
10000: fail obscured

I have no clue where the new boundary 1 lies, so I’ll use bisection to find it:

bisect 1
highest value for 'pass': 10
lowest value for 'fail empty': 5690
2850 characters loaded on the clipboard

This test also failed, so I recorded the result and bisect again.

fail empty
2850 recorded as fail empty

bisect 1
highest value for 'pass': 10
lowest value for 'fail empty': 2850
1430 characters loaded on the clipboard

To complete the bisection, I repeated this process: do the next test, record the result, and bisect again. This was the end result:

bisect 1
Boundary found!
highest value for 'pass': 1024
lowest value for 'fail empty': 1025

10: pass
720: pass
897: pass
986: pass
1008: pass
1019: pass
1024: pass
--boundary 1
1025: fail empty
1027: fail empty
1030: fail empty
1075: fail empty
1430: fail empty
2850: fail empty
5690: fail empty
--boundary 2
5691: fail obscured
10000: fail obscured

I reported the two bugs to the Audacity project, and called it a success. Note: this was a case of a moving boundary – on two previous days of testing, the failure point was at 1028 and 1299, though within each test session I didn’t observe the boundary moving.

Besides adding the rest of the perlclip features and porting to Windows, the next things I’d like to implement for testclip are snapping the bisection to powers of 2 and multiples of 10, since bugs tend to lurk in those spots, and finishing a feature that can bisect on integer values in addition to counterstrings.

For more about perlclip, see “A Look at PerlClip” (free registration required).

photo credit: Wajahat Mahmood

Amplifying the Comment Challenge


An ongoing topic on Twitter is resonating with me – the #CommentChallenge, described in this blog post by Kristīne Corbus – Comment Challenge. The challenge is basically to leave a comment on at least one blog post a week that’s relevant to your work.

This is something I tend to do anyway. It doesn’t matter if I’m reading an article, book, discussion forum, or blog post, or listening to a podcast or any other media, I’m always looking for a way to engage with the author. If an author gives me something useful, I have an opportunity to make it a richer experience with the author’s help, and possibly strengthen my network. Also, if I expect I may be posting a comment, I find that I read the information more carefully and I’m therefore more likely to retain it.

When I’m reading a blog, the form of my feedback may be a blog comment, but that’s not generally a great platform for an extended discussion. So if I really want to get into the topic and I know where the author hangs out online, I may start a discussion elsewhere. I’ll probably also post some sort of comment on the blog, because that helps to show the public that the blog has engaged readers (especially if there are no comments yet), which helps the author. A habit that serves me well is just trying to be helpful.

These are the types of situations where I tend to offer comments:

  • When I have a question about something the author said or a closely related topic, something I’d really like to learn – either the author’s opinion, or facts that are hard to find elsewhere.
  • When the author asked a question and I have potentially a useful answer.
  • When I have something to add to what the author said that I think will be highly valuable, even if the author didn’t ask for this type of feedback. I try not to do this very often.
  • When I disagree with something the author said. I think carefully before I do this. Doing this can often earn the respect of the author, and we’re both likely to learn something from the exchange, but when done in the wrong way, it can damage both my relationship with the author and my public reputation. I won’t try to elaborate on all the subtleties here. Often I just ask a question instead of stating directly that the author is wrong. I may find out that I misunderstood something they said, and I don’t actually disagree with them.
  • When I want to give kudos to the author for making an important point or for making a point particularly well. This type of feedback is less useful than the rest, so it’s best to combine it with one of the items above.

I still don’t comment on everything I read – only those things that I have a useful reaction to that I can share.

My challenge to myself is a bit different from the Comment Challenge. I tend to let my learning habit fizzle, so that I stop taking the time to read or otherwise learn new things. So my challenge is to expose myself to new things every week. When I do that, the comments will naturally follow.

The Ethics of Testing a Public Server

I like to hone my testing skills by trying different techniques. Sometimes the project I happen to be working on serves well as a sandbox for this, but not always. I also like to write about testing techniques using examples that other people can try. So it’s convenient to have an easily accessible application that I can write about.

I’ve been working on generating test data like long strings and large numbers with the venerable perlclip tool and a partial perlclip port to Ruby that I call testclip. I’m curious what you think about the ethics of testing in each of these real situations below.

1) Sorry, Wikipedia

I was having a discussion with a contact at Wikipedia, and I wanted to illustrate how I use bisection with long strings to isolate a bug. I wanted to find a bug on Wikipedia itself, so I tested its search feature. I considered the risks of testing on their production system – though long strings are fairly likely to find a bug, I couldn’t remember ever seeing them cause a catastrophic failure. So I judged that it was appropriate to continue. I think my contact was aware that I was testing it, but I didn’t explain the risks and he didn’t grant explicit permission.

Wikipedia gave me an ideal example, with a minor failure on a moderately long search string, and a more severe error with a much longer string (I went up to about 10,000 characters). I started writing up my analysis. As I went back to reproduce a few of the failures again, I noticed a new failure mode I hadn’t noticed before. Rather than isolate this new failure, I decided to stop testing. It seemed unlikely that my testing was related to this, but I wanted to make sure.

When I got in touch with my contact at Wikipedia, I found out that I had caused a major worldwide outage in their search feature. I did a lot of reflection after that – I really regretted causing this damage to a production system.

Was it ethical for me to run these tests?

2) Please test my site

I listened in to the virtual STAR East 2016 conference, which had a Test Labs activity that was accessible for virtual participants. I didn’t really understand what the activity was, but I did see that we were invited to test a particular open source application, CrisisCheckin, and report bugs on GitHub. An instance of the server was set up for testing. I used this as motivation to add a feature to testclip to bisect on an integer value in addition to the length of a counterstring.

It was nice to have a test instance of the system. I still considered the possibility that my testing could cause an outage that would affect the other people who were using the test instance. I decided to take the risk. The long strings I tested with made all similar types of data slightly more difficult for all users to read on the page, and in some cases the user interface didn’t provide a way to delete the data, so I did have a small impact on the shared system. I didn’t cause any outages that I was aware of.

There were instructions on GitHub for setting up a local instance of the software, which would be ideal in terms of not interfering with anyone else’s use of the site, but I chose not to take the time to do that.

Would you agree that my testing in this case was ethical?

3) It’s popular, so I’m picking on it

I’m working on writing an example usage of perlclip now, where I chose to pick on the main Google search field. I tested with a search string up to 1000 characters long, which finds a minor bug, but doesn’t seem to affect the availability of the system.

Is it ethical for me to do this testing, and publish something that encourages others to do the same?

A common reaction to these questions I’ve heard is that it’s the responsibility of the owners of the web site to make the site robust, so it’s not my fault if I’m able to do something though the user interface that breaks it. I don’t think it’s that simple.

I perused the Code of Ethics for the Association for Software Testing, and I didn’t see anything that directly addresses this question, though it’s clear on what to do when we do cause harm. At least for example 1 and 3 here, I’m not using these services for the purposes they were intended for. The Terms of Service for Google don’t actually say that I have to use it for the intended purpose. The Wikipedia Terms of Use, though, do talk about testing directly, which is expressly allowed in some situations. This testing is not allowed if it would “…unduly abuse or disrupt our technical systems or networks.” The terms also don’t allow disrupting the site by “placing an undue burden on a Project website.” So clearly it’s bad to cause an outage, but difficult to assess the risk in advance of an outage happening.

It’s much more clear that it’s not okay to conduct security testing without explicit permission. Security testing includes looking for denial of service vulnerabilities. But my intentions for doing long string testing generally aren’t to find vectors for a denial of service attack, even if that’s what happened in one case.

So how much caution is warranted to mitigate the risks of long string testing on production servers?

If the conclusion is that we should never test with long strings in production (at least without permission), then we have to look for safe places to practice our testing skills. Running a personal instance of an application server is one option, but that isn’t easy for a everyone to do. Another option is having a public sandbox that we can access, as we have with CrisisCheckin. There are several cases of servers set up for educational purposes, either associated with exercises in a book or with a training class. Many of those, though, are only intended for customers who bought the book or the class. I think I’ll shift my focus to native applications that run locally and are easy to install. My head is in the web so much, I forget that there is such a thing as a local application. 🙂

Podcasts I’m listening to

I’m going to start writing about software testing again; an easy way to jump in is to discuss podcasts. I’ve been listening to a lot of podcasts lately to hone my technical skills.

The podcast I’m most familiar with that’s directly related to software testing is AB Testing, hosted by Alan Page and Brent Jensen from Microsoft. These guys are right on the leading edge, and they go into a good amount of detail about how they do what they do. They aren’t afraid to express opinions about what they don’t like, too.

I’ve also recently started listening to TestTalks by Joe Colantonio and Testing In The Pub by Stephen Janaway and Dan Ashby.

On the development side, I like Developer on Fire, hosted by David Rael. David has amassed an impressive range of interviewees. I’ve also been listening to Agile for Humans, from Ryan Ripley, Don Gray, Tim Ottinger, Amitai Schlair, and Jason Tice.

I’m trying to become more knowledgeable about software security, with the help of the Silver Bullet Podcast from my former co-worker Gary McGraw.

What else can you add to the list?


Roadside Attractions

Lucy stands stall in front of an 80 foot tall Van Gogh replica.

Lucy stands stall in front of an 80 foot tall Van Gogh replica.

On our recent road trip, I was introduced to my wife’s tradition of looking for roadside attractions along the way. Thanks to Roadside America, we had a wide variety of things to see that could break the monotony of the drive.

Fifi in four states at once

Fifi in four states at once

Some of the attractions had historical value, like Custer’s Last Stand: Little Big Horn (Crow Agency, Montana), Four Corners Monument (near Shiprock, New Mexico), and Billy the Kid’s grave (Fort Sumner, New Mexico). A few we would have done even if we hadn’t found them on Roadside America, like the Seattle Underground Tour (Seattle, Washington). The Berkeley Pit Overlook (Butte, Montana), a former pit mine now holding a toxic lake, was a surprising but informative stop. Even the Statue of Victim of Sewage Tank Collapse (Spokane, Washington) taught us a bit of local history.

Yep, it looks like a milk bottle

Yep, it looks like a milk bottle

A few of the attractions only merited a glance out the window as we drove by, like the Plumber Guy (Moab, Utah), the Church of God-Zilla, Zilla, Washington, the Milk Bottle Building, (Spokane, Washington), the Center of the Universe (Wallace, Idaho), and Our Lady of the Rockies (Butte, Montana).

There were inevitably a few disappointments. We were unable to get our car to roll uphill on Gravity Hill (Salt Lake City, Utah). Excitement built up to a crescendo as we approached the Wonder Tower (Genoa, Colorado), but it was closed indefinitely by the time we arrived.

A house carved into a mountain

A house carved into a mountain

Other worthwhile distractions on trip were the World’s Longest Aerial Tramway – really more than just a roadside attraction (Albuquerque, New Mexico), Hole N’ The Rock (Moab, Utah), Spider VW Bug (Lexington, Oklahoma), Teapot Dome Gas Station (Zilla, Washington), Dick and Jane’s Spot (Ellensburg, Washington), Fremont Troll (Seattle, Washington), World’s Largest Easel (Goodland, Kansas), Glenn Goode’s Big People (Gainesville, Texas).

My lovely wife even submitted one of the attractions to Roadside America – Danger: Falling Cows (Manson, Washington).  But there was one roadside attraction that clearly stood out as her favorite. Which one? The World’s Largest Ball of Twine (Cawker City, Kansas). Now we’ve been there, done that.

That's one big ball of twine

That’s one big ball of twine

Restaurants on the Road


101_3846Several of my fond memories from our recent road trip are from the restaurants we visited. A few of them were planned in advance, but most were found serendipitously as we looked for something with local flavor. We loaded up on snack food so we could avoid at least one stop each day, and that made the places we did stop at more appreciated.

Big Apple Deli, Snyder, TX

I was surprised to find something named “Big Apple” in West Texas. This was a nice deli, and we enjoyed being on the patio so we could get the dog and the chicken out of the car.

Mykonos Cafe & Taverna, Albuquerque, NM 

We saw Greek in the GPS and made a beeline. This looked like a very nice space. Too nice for the boisterous children we had with us, and we had places to go. We got a bunch of food to go and took it to the top of Sandia Peak where we had a picnic with great food and a great view.

Idaho Joe’s, Twin Falls, ID 

When we reached Idaho, we decided that we simply must have potatoes. Idaho Joe’s sounded like the perfect place. Again, we had places to go. We ordered a selection of potato items to go, plus something I recall that they called scones but looked like frybread to me (really good stuff). We devoured it at Shoshone Falls nearby.

Bar 14 Ranch House Restaurant, Ellensburg, WA

We remember this stop because it was so nice to get out of the car and sit in a real chair. It was a welcome break after snacking in the car all day.

Wind River, Ellensburg, WA

The sign promised a dog run, so we walked here after we finished at the Bar 14. The dog wasn’t impressed until the staff invited her inside the store. The ice cream was great, especially with an added shot of espresso. We grabbed a few specialty beers from the cooler that we saved for later.

Falafel King, Seattle, WA 

My wife lingered at the smell emanating from this restaurant as we walked down the sidewalk from the map store. Most of the people in our group at the time weren’t excited about Mediterranean food, but I encouraged her to get the shawarma anyway. Thankfully, she couldn’t finish all of it, so I got to help her with it, and it was so good my mouth still waters thinking about it.

Ray’s Boathouse, Seattle, WA 

Ray’s hosted a wedding we attended, which is also where dinner was served. We enjoyed the view of the sunset over the Puget Sound. When they set down a plate full of steak in from of our meat-loving 5-year old, his eyes got as big as the plate. Delicious food, and the staff was very friendly and helpful. Really top-notch service.

BBQ in the Vineyard Restaurant, Chelan, WA

This one is a family tradition, an open-air restaurant at the Lake Chelan Winery. It’s nice to sit among the vines while the kids frolic on the hillside. Of course, we had to try a few bottles of wine. They didn’t complain once that we bought the chicken with us, and several of the other guests enjoyed visiting Fifi. One lost a bet, insisting that it must be a turkey. It’s funny how people don’t recognize a chicken when it’s between the newborn egg shape and full grown.

Red Light Garage, Wallace, ID

We visited Wallace to see the metal plate in the middle of the street declaring that it’s the center of the universe. While looking for a gas station, we spotted the Red Light Garage restaurant with a spaceship in the parking lot. We enjoyed some huckleberry ice cream on the patio while the pets stretched their legs. The kids enjoyed sitting in the spaceship.

Fred’s Mesquite Grill, Butte, MT

We arrived shortly before closing, and the hostess discouraged us from sitting on the patio. Did she somehow know of our habit of bringing the pets out of the car? But it was hot inside, and almost everyone was on the patio, so we insisted, though we decided not to bring the animals over. There were compliments for the food all around the table. The most memorable thing, at least for my daughter, was the busboy who kept stealing glances at her.

I-70 Diner, Flagler, CO

We were headed toward a Subway, but stopped short when we saw this beautiful diner. I don’t remember much about my meal, except cream gravy was involved. And the rhubarb pie was great. It was fun to watch the pink Cadillac on top of the pole out front, turning so slowly that we weren’t sure for a while whether it was moving at all. I read the history of the restaurant that was included in the menu, but the writing was just convoluted enough that it was hard for me to piece together.

Thinking over all of our culinary experiences makes me want to hit the road again!