The Ethics of Testing a Public Server

I like to hone my testing skills by trying different techniques. Sometimes the project I happen to be working on serves well as a sandbox for this, but not always. I also like to write about testing techniques using examples that other people can try. So it’s convenient to have an easily accessible application that I can write about.

I’ve been working on generating test data like long strings and large numbers with the venerable perlclip tool and a partial perlclip port to Ruby that I call testclip. I’m curious what you think about the ethics of testing in each of these real situations below.

1) Sorry, Wikipedia

I was having a discussion with a contact at Wikipedia, and I wanted to illustrate how I use bisection with long strings to isolate a bug. I wanted to find a bug on Wikipedia itself, so I tested its search feature. I considered the risks of testing on their production system – though long strings are fairly likely to find a bug, I couldn’t remember ever seeing them cause a catastrophic failure. So I judged that it was appropriate to continue. I think my contact was aware that I was testing it, but I didn’t explain the risks and he didn’t grant explicit permission.

Wikipedia gave me an ideal example, with a minor failure on a moderately long search string, and a more severe error with a much longer string (I went up to about 10,000 characters). I started writing up my analysis. As I went back to reproduce a few of the failures again, I noticed a new failure mode I hadn’t noticed before. Rather than isolate this new failure, I decided to stop testing. It seemed unlikely that my testing was related to this, but I wanted to make sure.

When I got in touch with my contact at Wikipedia, I found out that I had caused a major worldwide outage in their search feature. I did a lot of reflection after that – I really regretted causing this damage to a production system.

Was it ethical for me to run these tests?

2) Please test my site

I listened in to the virtual STAR East 2016 conference, which had a Test Labs activity that was accessible for virtual participants. I didn’t really understand what the activity was, but I did see that we were invited to test a particular open source application, CrisisCheckin, and report bugs on GitHub. An instance of the server was set up for testing. I used this as motivation to add a feature to testclip to bisect on an integer value in addition to the length of a counterstring.

It was nice to have a test instance of the system. I still considered the possibility that my testing could cause an outage that would affect the other people who were using the test instance. I decided to take the risk. The long strings I tested with made all similar types of data slightly more difficult for all users to read on the page, and in some cases the user interface didn’t provide a way to delete the data, so I did have a small impact on the shared system. I didn’t cause any outages that I was aware of.

There were instructions on GitHub for setting up a local instance of the software, which would be ideal in terms of not interfering with anyone else’s use of the site, but I chose not to take the time to do that.

Would you agree that my testing in this case was ethical?

3) It’s popular, so I’m picking on it

I’m working on writing an example usage of perlclip now, where I chose to pick on the main Google search field. I tested with a search string up to 1000 characters long, which finds a minor bug, but doesn’t seem to affect the availability of the system.

Is it ethical for me to do this testing, and publish something that encourages others to do the same?

A common reaction to these questions I’ve heard is that it’s the responsibility of the owners of the web site to make the site robust, so it’s not my fault if I’m able to do something though the user interface that breaks it. I don’t think it’s that simple.

I perused the Code of Ethics for the Association for Software Testing, and I didn’t see anything that directly addresses this question, though it’s clear on what to do when we do cause harm. At least for example 1 and 3 here, I’m not using these services for the purposes they were intended for. The Terms of Service for Google don’t actually say that I have to use it for the intended purpose. The Wikipedia Terms of Use, though, do talk about testing directly, which is expressly allowed in some situations. This testing is not allowed if it would “…unduly abuse or disrupt our technical systems or networks.” The terms also don’t allow disrupting the site by “placing an undue burden on a Project website.” So clearly it’s bad to cause an outage, but difficult to assess the risk in advance of an outage happening.

It’s much more clear that it’s not okay to conduct security testing without explicit permission. Security testing includes looking for denial of service vulnerabilities. But my intentions for doing long string testing generally aren’t to find vectors for a denial of service attack, even if that’s what happened in one case.

So how much caution is warranted to mitigate the risks of long string testing on production servers?

If the conclusion is that we should never test with long strings in production (at least without permission), then we have to look for safe places to practice our testing skills. Running a personal instance of an application server is one option, but that isn’t easy for a everyone to do. Another option is having a public sandbox that we can access, as we have with CrisisCheckin. There are several cases of servers set up for educational purposes, either associated with exercises in a book or with a training class. Many of those, though, are only intended for customers who bought the book or the class. I think I’ll shift my focus to native applications that run locally and are easy to install. My head is in the web so much, I forget that there is such a thing as a local application. 🙂

Podcasts I’m listening to

I’m going to start writing about software testing again; an easy way to jump in is to discuss podcasts. I’ve been listening to a lot of podcasts lately to hone my technical skills.

The podcast I’m most familiar with that’s directly related to software testing is AB Testing, hosted by Alan Page and Brent Jensen from Microsoft. These guys are right on the leading edge, and they go into a good amount of detail about how they do what they do. They aren’t afraid to express opinions about what they don’t like, too.

I’ve also recently started listening to TestTalks by Joe Colantonio and Testing In The Pub by Stephen Janaway and Dan Ashby.

On the development side, I like Developer on Fire, hosted by David Rael. David has amassed an impressive range of interviewees. I’ve also been listening to Agile for Humans, from Ryan Ripley, Don Gray, Tim Ottinger, Amitai Schlair, and Jason Tice.

I’m trying to become more knowledgeable about software security, with the help of the Silver Bullet Podcast from my former co-worker Gary McGraw.

What else can you add to the list?


Roadside Attractions

Lucy stands stall in front of an 80 foot tall Van Gogh replica.

Lucy stands stall in front of an 80 foot tall Van Gogh replica.

On our recent road trip, I was introduced to my wife’s tradition of looking for roadside attractions along the way. Thanks to Roadside America, we had a wide variety of things to see that could break the monotony of the drive.

Fifi in four states at once

Fifi in four states at once

Some of the attractions had historical value, like Custer’s Last Stand: Little Big Horn (Crow Agency, Montana), Four Corners Monument (near Shiprock, New Mexico), and Billy the Kid’s grave (Fort Sumner, New Mexico). A few we would have done even if we hadn’t found them on Roadside America, like the Seattle Underground Tour (Seattle, Washington). The Berkeley Pit Overlook (Butte, Montana), a former pit mine now holding a toxic lake, was a surprising but informative stop. Even the Statue of Victim of Sewage Tank Collapse (Spokane, Washington) taught us a bit of local history.

Yep, it looks like a milk bottle

Yep, it looks like a milk bottle

A few of the attractions only merited a glance out the window as we drove by, like the Plumber Guy (Moab, Utah), the Church of God-Zilla, Zilla, Washington, the Milk Bottle Building, (Spokane, Washington), the Center of the Universe (Wallace, Idaho), and Our Lady of the Rockies (Butte, Montana).

There were inevitably a few disappointments. We were unable to get our car to roll uphill on Gravity Hill (Salt Lake City, Utah). Excitement built up to a crescendo as we approached the Wonder Tower (Genoa, Colorado), but it was closed indefinitely by the time we arrived.

A house carved into a mountain

A house carved into a mountain

Other worthwhile distractions on trip were the World’s Longest Aerial Tramway – really more than just a roadside attraction (Albuquerque, New Mexico), Hole N’ The Rock (Moab, Utah), Spider VW Bug (Lexington, Oklahoma), Teapot Dome Gas Station (Zilla, Washington), Dick and Jane’s Spot (Ellensburg, Washington), Fremont Troll (Seattle, Washington), World’s Largest Easel (Goodland, Kansas), Glenn Goode’s Big People (Gainesville, Texas).

My lovely wife even submitted one of the attractions to Roadside America – Danger: Falling Cows (Manson, Washington).  But there was one roadside attraction that clearly stood out as her favorite. Which one? The World’s Largest Ball of Twine (Cawker City, Kansas). Now we’ve been there, done that.

That's one big ball of twine

That’s one big ball of twine

Restaurants on the Road


101_3846Several of my fond memories from our recent road trip are from the restaurants we visited. A few of them were planned in advance, but most were found serendipitously as we looked for something with local flavor. We loaded up on snack food so we could avoid at least one stop each day, and that made the places we did stop at more appreciated.

Big Apple Deli, Snyder, TX

I was surprised to find something named “Big Apple” in West Texas. This was a nice deli, and we enjoyed being on the patio so we could get the dog and the chicken out of the car.

Mykonos Cafe & Taverna, Albuquerque, NM 

We saw Greek in the GPS and made a beeline. This looked like a very nice space. Too nice for the boisterous children we had with us, and we had places to go. We got a bunch of food to go and took it to the top of Sandia Peak where we had a picnic with great food and a great view.

Idaho Joe’s, Twin Falls, ID 

When we reached Idaho, we decided that we simply must have potatoes. Idaho Joe’s sounded like the perfect place. Again, we had places to go. We ordered a selection of potato items to go, plus something I recall that they called scones but looked like frybread to me (really good stuff). We devoured it at Shoshone Falls nearby.

Bar 14 Ranch House Restaurant, Ellensburg, WA

We remember this stop because it was so nice to get out of the car and sit in a real chair. It was a welcome break after snacking in the car all day.

Wind River, Ellensburg, WA

The sign promised a dog run, so we walked here after we finished at the Bar 14. The dog wasn’t impressed until the staff invited her inside the store. The ice cream was great, especially with an added shot of espresso. We grabbed a few specialty beers from the cooler that we saved for later.

Falafel King, Seattle, WA 

My wife lingered at the smell emanating from this restaurant as we walked down the sidewalk from the map store. Most of the people in our group at the time weren’t excited about Mediterranean food, but I encouraged her to get the shawarma anyway. Thankfully, she couldn’t finish all of it, so I got to help her with it, and it was so good my mouth still waters thinking about it.

Ray’s Boathouse, Seattle, WA 

Ray’s hosted a wedding we attended, which is also where dinner was served. We enjoyed the view of the sunset over the Puget Sound. When they set down a plate full of steak in from of our meat-loving 5-year old, his eyes got as big as the plate. Delicious food, and the staff was very friendly and helpful. Really top-notch service.

BBQ in the Vineyard Restaurant, Chelan, WA

This one is a family tradition, an open-air restaurant at the Lake Chelan Winery. It’s nice to sit among the vines while the kids frolic on the hillside. Of course, we had to try a few bottles of wine. They didn’t complain once that we bought the chicken with us, and several of the other guests enjoyed visiting Fifi. One lost a bet, insisting that it must be a turkey. It’s funny how people don’t recognize a chicken when it’s between the newborn egg shape and full grown.

Red Light Garage, Wallace, ID

We visited Wallace to see the metal plate in the middle of the street declaring that it’s the center of the universe. While looking for a gas station, we spotted the Red Light Garage restaurant with a spaceship in the parking lot. We enjoyed some huckleberry ice cream on the patio while the pets stretched their legs. The kids enjoyed sitting in the spaceship.

Fred’s Mesquite Grill, Butte, MT

We arrived shortly before closing, and the hostess discouraged us from sitting on the patio. Did she somehow know of our habit of bringing the pets out of the car? But it was hot inside, and almost everyone was on the patio, so we insisted, though we decided not to bring the animals over. There were compliments for the food all around the table. The most memorable thing, at least for my daughter, was the busboy who kept stealing glances at her.

I-70 Diner, Flagler, CO

We were headed toward a Subway, but stopped short when we saw this beautiful diner. I don’t remember much about my meal, except cream gravy was involved. And the rhubarb pie was great. It was fun to watch the pink Cadillac on top of the pole out front, turning so slowly that we weren’t sure for a while whether it was moving at all. I read the history of the restaurant that was included in the menu, but the writing was just convoluted enough that it was hard for me to piece together.

Thinking over all of our culinary experiences makes me want to hit the road again!

A Bed, a Shower, and a Password, Please

Whether staying in a hotel or a house, one of the first necessities to take care of is figuring out how to get wi-fi access. It can be as important as the bed and the shower.

I have to admit that I don’t have a smart phone with cellular data. I carried a Blackberry for a previous employer that was nice to have for simple tasks, especially when there was no wi-fi nearby, and I’m having fun now with a borrowed iPad with cellular data. But that’s not good enough when I’m trying to move a lot of data, or in situations like I’m in now, where the house I’m staying in is nestled between hills that completely block all cell signals.

Internet access at a hotel can be an adventure. Often it seems that so many people are sharing the high-speed connection that it’s not high speed for anyone. I’ve done some software testing using such terrible hotel networks that I reproduced bugs there that I never saw anywhere else. At least at a hotel, I can easily ask at the front desk or look for instructions in the room for how to get connected. When I’m staying at someone’s house, the situation can be more delicate.

When I arrange to stay at the house of a friend or family member, I don’t generally ask in advance about Internet access. It seems impolite, when I’m getting free accommodation and maybe even free meals, to expect to use their Internet connection, too. My hosts rarely volunteer the details about their wi-fi network, which usually requires a password. But it would hardly affect them at all to share a few megabytes with me, would it? Often it wouldn’t, but where I’m at now, they say they’re nearing their monthly quota of data on their satellite Internet service, and they risk getting their bandwidth throttled if they exceed the quota, so the wi-fi is turned off most of the time.

I’m reminded of the time I had house guests from Denmark. They were keen to get Internet access to be in touch with home. For some reason, I needed to move my Uverse router, and when I plugged it back in, I didn’t get it right. We were unplugged from the Internet for a few days, which must have been an annoyance to them, though they didn’t complain. I felt I had failed as a host.

So you’ll see this post the next time we go on an excursion beyond the hills. Meanwhile, I’m learning that I can survive with only a land line phone to connect me to the outside world.

What do you think – should house guests expect their hosts to share their Internet service, and fork over the password as part of showing where the bedroom and bathroom are?

Traveling with a Large Family

Finding overnight accommodations for more than four people has been a challenge. A majority of hotel rooms have one or two beds, accommodating up to four people if two people are willing to share each bed.

To accommodate more than four people, the option that hotels seem to prefer is for you to get multiple rooms. This presents numerous problems. Of course, this is much more expensive. It would be nice to have adjoining rooms with a door you can open between them. It seems that adjoining rooms are getting more difficult to find, especially on-line. So there’s a good chance that two rooms would only be connected by a hallway, and if the hotel is mostly full, your rooms might not even be all on the same floor. There is no way that I’ve seen to get a reservation that guarantees that multiple rooms will be near each other.

Hotels often require an adult to be in each room when there is a multiple room reservation. So a couple traveling with children would have to stay apart from each other, and a single adult traveling with several children would not have an option to use more than one room.

So assuming everyone is willing to share a bathroom (except for the rare case of a suite with more than one bathroom), I prefer to get everyone into one room. Online reservation systems do ask how many people are staying, but then might offer a room with only one bed for six people, without explaining whether there are sofa beds or cots available (and usually don’t explain the extra charges for additional bedding).

I have enjoyed using SixSuitCaseTravel to find information about hotels for larger groups. One concept you may find there is “room for 6, book for 5”, if you have small children you can fit two to a bed that the hotel says should only fit one person.

Can we open up additional viable hotel options online if we don’t indicate how people are really staying in the room? I don’t want to cheat the hotel if the prices are based on how many people are in the room. Maybe there are fire codes that are relevant to how many people are in a room? But I know there are cases where we find a room that would make my group happy but we have to get creative with the search parameters to find it.

What’s your experience with trip planning for a group of more than four?

Show Me the Chicken


Seems that when you travel with a chicken, you shouldn’t do it halfway. We went to a friend’s wedding last night. I had joked with our friend that we needed to arrange for another guest at dinner, with an insectivore meal. Practically all of the other guests had already heard that we were traveling with a chicken. And our friend was genuinely disappointed that we didn’t bring the chicken to his wedding.

I’m sitting at Firestone right now. I’ve gotten to know them pretty well here over the last few days as we got a leak in the cooling system diagnosed, waited for a part to arrive, and in the middle of it all, retrieved the car temporarily so we could get to a wedding. They’re pretty friendly here, and we got to talking, so of course they got to hear about the chicken. Every time they’ve seen us since then, they’ve asked whether we brought the chicken so they could see her. Alas, logistics have required the chicken to stay cooped up at the house for a few days. At least I could show them the chicken on my blog.

So be warned, if you talk about your traveling chicken, you might be asked to produce the chicken.

Fifi Contemplates the Open Road



While barreling down the highway, we decided to put the chicken on the dashboard so she could see the wide, wide world out there. I swear she had a look of awe on her face, slowly trying to absorb the magnificent landscape, and how big it all was, and how fast it was flying by.

She wanted to come down soon after. It was just so much for her little brain to handle.

Talk to Strangers

We were taking a ferry from Seattle to the Kitsap Penninsula today. Toward the end of the ride, a lone man approached my wife and me to comment on the beautiful sunset. He seemed friendly, and we chatted for a while. He mentioned a job in telecommunications, and I said he looked more like the artist type to me. He was dressed in an eclectic jacket with big buttons, a V-neck shirt, and a necklace that looked like a claw of some sort. His hair was, well, big, and further exaggerated by the stiff wind. Sure enough, he’s a musician.

We talked about mandolin fingering and the bizarre string layout on ukuleles, and then one of the kids made a disparaging remark about my kazoos. Well, that led us down a path of kazobos, wazoos, and electric kazoo pickups. As we parted, I promised to send him information about the Kazoobie electric kazoo.

I was dressed plainly – jeans and a T-shirt. I was even giving my hiking boots a well-deserved rest for the day, and was wearing my sneakers. It’s hard to dress artistically when you have to pack light because of limited luggage room. So our new artist friend was pleasantly surprised after he first approached us to find us talking about kazoo lore and chickens on a road trip.

The thing that stuck with me was when he encouraged us to “talk to strangers”. It turns out that’s the name of his album. I have the feeling that’s a key part of his philosophy of life.

(The stranger’s name is Joe Abrams.)