entries friends calendar user info Bryant's Homepage Previous Previous Next Next
Bryant - Amazon Troll Busting
bryant
[info]bryant
Add to Memories
Share this!
Amazon Troll Busting
The original post which prompted this one is now inaccessible accessible again. C'est la vie.




Hopefully you weren't thinking of this as a news post, but just in case:

http://blog.seattlepi.com/amazon/archives/166329.asp

"This is an embarrassing and ham-fisted cataloging error for a company that prides itself on offering complete selection," says the Amazon spokesperson. It's not as complete an explanation as I wish we could have, but it is not any sort of attempt to save face.




Oooh, new update!

There is a "report this content as inappropriate" box at the bottom of at least some Amazon pages. It doesn't go to the URL claimed, though. It goes to http://www.amazon.com/gp/product/features/fiona-feedback-email.html.

It is absolutely fascinating that after I screwed up and completely missed that link, nobody double-checked me. See philosophical pontificates lower down in this post.

(And nah, I still don't buy the troll -- the URL inconsistency is still wacky, and it's too easy to say "sure, it's not there because the functionality was removed." Especially since it wasn't. I do think he was making a point about trust on the Internet, and I do still think it's a good one.)




Oh, OK, one pair of links:

http://mikedaisey.com/
http://www.lilithsaintcrow.com/journal/2009/04/idosyncratic-code-amazonfail/

Mike Daisey used to work at Amazon, so it's not impossible that he has contacts inside the company who'd tell him the truth. I don't know of any motivation he has to make shit up, although he does make his living from people buying tickets to his work. He is a friend of a friend who I trust, but I don't know him personally. The story is plausible but also lacks detail.

The above paragraph is the sort of analysis to which one ought to subject any of these random assertions.




There's a guy claiming he abused Amazon's reporting system in order to get GLBT books removed from the sales rank listings. Since I have some pretension to technical ability, I figured I'd give his claims a test run. (Edit: post now protected and unreadable, unless you're a community member.)

Summation: nope, you didn't do that, you liar you. Nice meta-troll, though.

Details:

a) The code is buggy; I can't get it to run as written. In particular, he uses the -dump parameter to links. That causes links to dump a formatted version of the document, which does not contain any URLs at all. (Edit: yeah, he gave Valleywag an explanation for this which does make sense.)

I went ahead and got a non-formatted version of the page he's grabbing for the sake of completeness, and ran his grep and sed statements on it. You don't actually get a pretty listing of product IDs from that. You could get one if you wrote better regexps, but the ones he's providing just don't work.

So let's say he was just, I don't know, obfuscating because he's lazy. It is entirely possible to get a list of Amazon product IDs by methods similar to the ones he posted. Onward!


Thought that was clear, but I don't mind making it crystal-clear: I'm convinced that you can get a list of Amazon product IDs using his code.

b) He says that URLs of the form http://www.amazon.com/ri/product-listing/ generate a complaint. However, if you go to a URL with that format, you get a 404 page. It's possible that Amazon just pulled that functionality this morning, but there's no sign of that URL in their help system that I can see after a quick once-over.

Edit: he is saying that the functionality was just pulled. So I dunno. Google caches of Amazon book pages don't show the link he's claiming was there, but they wouldn't if that functionality is dependent on being logged in. Anyone have an old screenshot of an Amazon page showing that?

Conclusion: troll! Reserving the right to change my mind if we get some real proof, but see lengthy philosophy note that follows.

Edit: it's an interesting bit of trolling, actually. He's piggybacking on some of the same tendencies that led the original story to turn into a Cause with a capital C. If you're not a geek, the Internet is just this weird magical place where stuff /happens/ and anything's likely if it's expressed with authority.

His post is even better because there's nothing inherently implausible about the idea of hiring a bunch of third-world sweatshop people to screw up a user-generated tagging/complaint system. Amazon doesn't appear to have made that mistake, but you need to check before you can be sure.

To make it even better, Twitter was the main vector of communication about the Amazon stuff. Twitter is lousy for any communication which takes more than 140 characters; it strips logic leaving us only with reputation capital. The #amazonfail tag got a lot of reputation capital, initially from upset people and later from sheer volume...

But you can't tell from a Twitter post whether or not something's authentic. You gotta do your own research and thinking. Some people do; lots of people don't. No matter what Amazon did or didn't do, intentionally or not, there is absolutely not enough evidence right now to draw any conclusions other than "it's bad that this happened." Our troll used the same transmission technique, because who's gonna take the time to read his post and think about his claims?

(Addition to that: seriously. Why do you believe him? Why do you believe me, for that matter?)

Good times. At some point we're going to have to figure out how to overcome a thousand years of conditioning: for a very long time, saying something loudly required a great deal of effort, so at least you knew someone really believed what they were saying. These days, no effort at all, but we still have that kneejerk reaction. (This is not an Internet problem. I blame LaserWriters.) Man, doesn't it just seem antiquated that Speaker's Corner used to be a huge deal and a symbol of free speech?

The really interesting thing about the troll is that he's right even if he didn't do it. The vulnerability he describes exists anywhere you make automated decisions based on third-party input.

Hah. See what happens when I get links? I pontificate.
Comments
Page 1 of 2
[1] [2]
From: (Anonymous) Date: April 13th, 2009 04:32 pm (UTC) (Link)
I rather thought so... thanks for checking, though: It's always better to know rather than assume.

Form a common sense standpoint, his story would not explain the two Amazon reps' response template about adult content.

Thank you.
(Replies frozen) (Thread) (Expand)
peitso From: [info]peitso Date: April 13th, 2009 04:44 pm (UTC) (Link)
The 'adult' content policy messages could have been a PR move because they really had no idea what was going on. No company ever tells the public that it's dealing with a massive crisis that hasn't surfaced yet--there's no need if it's not public. If that was the case, it was still a bad move.
(Replies frozen) (Parent) (Thread) (Expand)
(no subject) - [info]head58 Expand
(no subject) - [info]bernmarx Expand
(no subject) - [info]head58 Expand
(no subject) - [info]bernmarx Expand
(no subject) - (Anonymous) Expand
(no subject) - [info]bryant Expand
(no subject) - (Anonymous) Expand
(no subject) - [info]peitso Expand
(no subject) - [info]bernmarx Expand
(no subject) - [info]just_ruth Expand
(no subject) - [info]7kim_moon Expand
I can answer this one... - [info]gerrib Expand
Re: I can answer this one... - [info]atara Expand
Re: I can answer this one... - [info]azurelunatic Expand
(no subject) - [info]kibbles Expand
(no subject) - [info]bryant Expand
(no subject) - [info]atara Expand
(no subject) - [info]gerrib Expand
(no subject) - [info]gerrib Expand
(no subject) - [info]bryant Expand
From: (Anonymous) Date: April 13th, 2009 04:38 pm (UTC) (Link)
Thanks for clearing this up. Even if this guy's actions had some kind of effect there's no way one person would be responsible of it all.
(Replies frozen) (Thread)
From: (Anonymous) Date: April 13th, 2009 04:40 pm (UTC) (Link)
works for me.

merk@locke:~ [31/83]$ links --version
ELinks 0.10.6 (built on Sep 25 2007 18:50:54)

Features:
Standard, Fastmem, IPv6, gzip, bzip2, Cascading Style Sheets, Protocol (File, FTP, HTTP, NNTP, SMB, URI rewrite, User protocols),
SSL (GnuTLS), MIME (Option system, Mailcap, Mimetypes files), LED indicators,
Bookmarks, Cookies, Form History, Global History, Scripting (Lua, Perl)
(Replies frozen) (Thread)
bryant From: [info]bryant Date: April 13th, 2009 04:58 pm (UTC) (Link)
Well, sure. links is a real program that really does go out and fetch pages from the Web.

But if you run links as:

links -dump

You get a formatted version of the page; you don't get the raw HTML, which is what you need if you want to extract the links.

I mean, it's a side point, because it's obviously possible to write a script that'll extract every product ID on Amazon, but the code's undeniably buggy as posted.
(Replies frozen) (Parent) (Thread) (Expand)
(no subject) - [info]weev Expand
(no subject) - [info]weev Expand
(no subject) - [info]bryant Expand
(no subject) - [info]daid Expand
(no subject) - [info]daid Expand
(no subject) - [info]weev Expand
troubleinchina From: [info]troubleinchina Date: April 13th, 2009 04:46 pm (UTC) (Link)
Also, it doesn't account for all of the books that were taken down. Amongst other things, straight erotica also went down, and that wouldn't go down with a "I'm targeting GLTB books" code.
(Replies frozen) (Thread)
nonnycat From: [info]nonnycat Date: April 13th, 2009 04:50 pm (UTC) (Link)
He mentioned getting other people in on it for a focused attack. If they changed the parameters, that might explain it.

That the code doesn't work at all, though......
(Replies frozen) (Parent) (Thread)
From: (Anonymous) Date: April 13th, 2009 04:50 pm (UTC) (Link)
Re 4):

He does insert the product IDs after http://www.amazon.com/ri/product-listing/ like so:

http://www.amazon.com/ri/product-listing/0830823794

But still results in 404.
(Replies frozen) (Thread)
bryant From: [info]bryant Date: April 13th, 2009 04:53 pm (UTC) (Link)
Yeah, I tried that and got the 404.

If someone can come up with evidence that it worked prior to today I'll recant, because turning off a Web page is exactly how I'd turn off that capability without having to do a code push. But I wanna see some evidence first.
(Replies frozen) (Parent) (Thread)
jonthegm From: [info]jonthegm Date: April 13th, 2009 04:52 pm (UTC) (Link)
The code he POSTED didn't work... but you can immediately see that given a slightly more powerful perl or python script that you could do it without links.
(Replies frozen) (Thread)
bryant From: [info]bryant Date: April 13th, 2009 04:55 pm (UTC) (Link)
I do! Heck, I wrote that code in about five minutes while I was fooling around with this. I'm willing to believe that he screwed with it for some reason.

Buttt the automated complaint page doesn't exist, which is a bigger problem.
(Replies frozen) (Parent) (Thread) (Expand)
(no subject) - [info]jonthegm Expand
(no subject) - [info]bryant Expand
(no subject) - [info]lynxreign Expand
(no subject) - [info]bryant Expand
(no subject) - [info]lynxreign Expand
(no subject) - [info]bryant Expand
(no subject) - [info]lynxreign Expand
(no subject) - [info]sylvanstargazer Expand
(no subject) - [info]head58 Expand
(no subject) - [info]bryant Expand
(no subject) - [info]lynxreign Expand
(no subject) - [info]nnaylime Expand
(no subject) - [info]sylvanstargazer Expand
From: (Anonymous) Date: April 13th, 2009 04:54 pm (UTC) (Link)
Thanks for clearing this up. Even if this guy's actions had some kind of effect there's no way one person would be responsible of it all.
(Replies frozen) (Thread)
stardragonca From: [info]stardragonca Date: April 13th, 2009 05:00 pm (UTC) (Link)
Of course, young Archimedes just made himself liable to a whole lot of grief.
Especially if Amazon decides they need a good financial scapegoat, and lookee who just volunteered!
(Replies frozen) (Thread) (Expand)
From: (Anonymous) Date: April 13th, 2009 06:02 pm (UTC) (Link)
He's a troll. He doesn't care who gives him grief; he just thinks it's funny that people are getting so worked up. If Amazon wants to get any money out of him, well... I wish them luck in finding his real name.

I mean, I'm not condoning his actions, I just don't think there's a lot you can do to a guy who doesn't take anything seriously. Besides ignoring him, that is.
(Replies frozen) (Parent) (Thread) (Expand)
(no subject) - [info]ravan Expand
(no subject) - [info]stardragonca Expand
(no subject) - [info]ilcylic Expand
(no subject) - [info]bryant Expand
(no subject) - [info]ilcylic Expand
(no subject) - [info]bryant Expand
(no subject) - [info]sultmhoor Expand
(no subject) - [info]bryant Expand
(no subject) - [info]elettaria Expand
(no subject) - [info]digitalsidhe Expand
(no subject) - [info]stardragonca Expand
emeraldsedai From: [info]emeraldsedai Date: April 13th, 2009 05:00 pm (UTC) (Link)
Appreciate the knowledgeable counter-perspective. I did, in fact, read the post on [info]brutal_honesty, along with everything else that's been linked a zillion times on Twitter. (Honestly, the internet was FUN last night!), and though I can't vet the guy's code, my tentative conclusion matches yours: nothing I've read so far makes total sense of the facts.
(Replies frozen) (Thread)
bryant From: [info]bryant Date: April 13th, 2009 05:01 pm (UTC) (Link)
I really /want/ to know what happened, because running big Internet stuff happens to be my job. I am sad that I probably won't ever really know.

It's damned well a good lesson in the vulnerabilities of centralization, no matter what. Same applies to Google. Same applies to UPS and FedEx. Single points of failure, right?
(Replies frozen) (Parent) (Thread) (Expand)
(no subject) - (Anonymous) Expand
(no subject) - [info]emeraldsedai Expand
(no subject) - [info]bryant Expand
(no subject) - [info]shadowflyer Expand
sunfell From: [info]sunfell Date: April 13th, 2009 05:11 pm (UTC) (Link)
Could the troll have used Amazon's "Mechanical Turk" to propogate his scheme? We'll have to keep an eye on that to see if there will be any policy changes there. Talk about being blown up with one's own petard...

I like Amazon- warts-and-all, it's my fall back when local places can't or won't stock books I need. I'd hate to see it messed up by something like this. They're out to make money- it doesn't make sense that an internal policy could become so toxic.
(Replies frozen) (Thread)
bryant From: [info]bryant Date: April 13th, 2009 05:17 pm (UTC) (Link)
Sure. You'd be nuts to use the Turk, because then there's a money trail which Amazon can look at without a court order, though. :)
(Replies frozen) (Parent) (Thread) (Expand)
(no subject) - [info]sunfell Expand
tacky_tramp From: [info]tacky_tramp Date: April 13th, 2009 05:14 pm (UTC) (Link)

Here via a friend's link

I was hoping a coder would go over that! I continue to suspend judgment until Amazon comes forward with more information. I assume they're planning to explain what went down and how they'll prevent it from going down in the future.
(Replies frozen) (Thread)
drivingblind From: [info]drivingblind Date: April 13th, 2009 05:16 pm (UTC) (Link)
You've got some good points about the "think about it" element. At this point, since I pointed at one of the earlier posts about #amazonfail, I've felt compelled to post follow-ups for folks to track the evolution of perspective. I don't always have the skills to "vet" those follow-ups, but I'm happy you have (and is why your post here is the latest in the sequence of stuff I've helped circulate).

This sort of thing is going to continue to happen. I think of getting from post one through to this post of yours is the process, though, whereby the "thinking" you're talking about *happens*. I'm not sure there's a failure in that, so much as a success.
(Replies frozen) (Thread)
bryant From: [info]bryant Date: April 13th, 2009 05:20 pm (UTC) (Link)
And thanks for the tweet!

I may post in the next couple of days about the danger of the -fail suffix, while I'm at it, because... hm. I think at this point anything which gets the -fail tag benefits (or suffers) from the reputation capital of Racefail. It's a really easy way to slot an event into the paradigm of ordinary people being oppressed.

I am paranoid about memetic viruses and I feel like a dork about it. Sigh.
(Replies frozen) (Parent) (Thread) (Expand)
(no subject) - [info]drivingblind Expand
(no subject) - [info]bryant Expand
(no subject) - [info]cgranade Expand
(no subject) - [info]melopoeia Expand
Turtles all the way down! - [info]amberley Expand
Re: Turtles all the way down! - [info]bryant Expand
masgramondou From: [info]masgramondou Date: April 13th, 2009 05:16 pm (UTC) (Link)
I do think Amazon's algorithms have been gamed even if this troll claim doesn't stand up. See http://www.di2.nu/200904/13a.htm

It is quite possible that this is an architectural problem
(Replies frozen) (Thread)
bryant From: [info]bryant Date: April 13th, 2009 05:22 pm (UTC) (Link)
That is a completely awesome post. Thank you for sharing it!
(Replies frozen) (Parent) (Thread) (Expand)
(no subject) - (Anonymous) Expand
(no subject) - [info]bryant Expand
lydia_golis From: [info]lydia_golis Date: April 13th, 2009 05:17 pm (UTC) (Link)
I'm a geek who considers the internet a magical place where stuff just /happens/. I'm a poli-sci geek. ;)
(Replies frozen) (Thread)
bryant From: [info]bryant Date: April 13th, 2009 05:45 pm (UTC) (Link)
Well, that's fair. I don't understand parliamentary procedure. ;)
(Replies frozen) (Parent) (Thread) (Expand)
(no subject) - [info]stardragonca Expand
(no subject) - [info]emilytheslayer Expand
(no subject) - [info]stardragonca Expand
(no subject) - [info]emilytheslayer Expand
lynxreign From: [info]lynxreign Date: April 13th, 2009 05:32 pm (UTC) (Link)
What language was used for his code?
(Replies frozen) (Thread)
bryant From: [info]bryant Date: April 13th, 2009 05:33 pm (UTC) (Link)
Bash shell scripting. Nothing complex.
(Replies frozen) (Parent) (Thread)
bernmarx From: [info]bernmarx Date: April 13th, 2009 05:54 pm (UTC) (Link)
Speaking of people putting too much stock in authority interpretations, it's amazing to me how many people think that Ashlyn D with Member Services is sanctioned to make public announcements of Amazon corporate policy when she's probably a low-level (if not minimum-wage-making) drone just trying to do her job.
(Replies frozen) (Thread) (Expand)
msilverstar From: [info]msilverstar Date: April 13th, 2009 06:20 pm (UTC) (Link)
Yeah, she's probably just some poor soul working on the weekend and sending out automated replies. At least her last name isn't out. Bezos, on the other hand...
(Replies frozen) (Parent) (Thread)
(no subject) - (Anonymous) Expand
marcmagus From: [info]marcmagus Date: April 13th, 2009 06:02 pm (UTC) (Link)
Works perfectly with lynx -dump, however. Perhaps his system has links symlinked to lynx, or it was, as you suggest, a bit of obfuscation? The sed works as advertised, although it also leaves in some junk which should be stripped out from other URLs.

He mentions in his post that the functionality was removed this morning by Amazon.

I have know idea if it's true, but it seems plausible.

I find your discussion of Twitter amplifying loudness quite interesting.
(Replies frozen) (Thread)
bryant From: [info]bryant Date: April 13th, 2009 06:10 pm (UTC) (Link)
links symlinked to lynx is possible. I did notice that he used lynx for the second stage.

Amazon removed the functionality... I mean, maybe. Google has caches of Amazon pages, and there's no link like that, but if that functionality only appears when you're logged in you wouldn't expect to see a link in the cache.

So this comes down to the nub of the question:

Why would you believe some random guy on the Internet when there's no proof that he did what he said he did? (And why would you believe me, for that matter?)

His story is definitely the exciting one. My claims are more satisfying if you're pre-inclined to believe that trolls are assholes. Neither of those things makes either of us right; neither of them are proof.

In a very real sense I don't think it matters if he's telling the truth or not. Even if Amazon didn't have a hole in their system, such holes are not uncommon. Possibly I shouldn't have said anything; people are more likely to believe it's a real problem if they believe it's happened once...
(Replies frozen) (Parent) (Thread) (Expand)
(no subject) - [info]msilverstar Expand
(no subject) - [info]nonnycat Expand
(no subject) - [info]marcmagus Expand
(no subject) - [info]bryant Expand
Brilliant! - [info]p_sunshine Expand
(no subject) - [info]bernmarx Expand
Security holes - (Anonymous) Expand
Re: Security holes - [info]bryant Expand
jonquil From: [info]jonquil Date: April 13th, 2009 06:03 pm (UTC) (Link)
Well done.

Dear Author quite conclusively demonstrates that the actual deletions were done on publisher metadata, not on user-supplied keywords. The known errors are predicted by errors in the metadata.

http://dearauthor.com/wordpress/2009/04/12/amazon-possibly-using-category-metadata-to-filter-rankings/#more-11485

When you add that to the evidence that books about disability sex having nothing to do with GLBTQ were targeted, it's pretty clear that the varlet in question is trolling.


Edited at 2009-04-13 06:05 pm (UTC)
(Replies frozen) (Thread)
bryant From: [info]bryant Date: April 13th, 2009 06:13 pm (UTC) (Link)
In the interests of being super-fair, it is possible to easily get the list of all books matching a specific chunk of metadata, and execute an attack on those books. I don't think Dear Author's work totally rules it out.

What does bug me about that post is that someone -- Amazon employee or otherwise -- did target specific metadata. Boo.
(Replies frozen) (Parent) (Thread) (Expand)
(no subject) - [info]jonquil Expand
asim From: [info]asim Date: April 13th, 2009 06:11 pm (UTC) (Link)
Thanks for doing the Hard Work. He wasn't passing my smell test with his code, and then I saw someone else try to run it, and fail...

Gah.
(Replies frozen) (Thread)
msilverstar From: [info]msilverstar Date: April 13th, 2009 06:16 pm (UTC) (Link)
Thanks, I had some friends taking it seriously, I'm linking to you for explanations.
(Replies frozen) (Thread)
From: (Anonymous) Date: April 13th, 2009 06:17 pm (UTC) (Link)
"It's possible that Amazon just pulled that functionality this morning"
They did.
(Replies frozen) (Thread)
bryant From: [info]bryant Date: April 13th, 2009 06:25 pm (UTC) (Link)
Did they?
(Replies frozen) (Parent) (Thread) (Expand)
(no subject) - [info]kynn Expand
immlass From: [info]immlass Date: April 13th, 2009 06:24 pm (UTC) (Link)
FWIW, I believe you because I've met you and I know what your professional skills are, and believe you could make the judgement in the post and would make it honestly due to our RL and online interactions. But that's just me.
(Replies frozen) (Thread)
bryant From: [info]bryant Date: April 13th, 2009 06:36 pm (UTC) (Link)
See, that's a good reason to believe me. He said modestly.

While I often disagree with Cory Doctorow on social issues, I think his vision of a formalized reputation capital system in which we believe people who our friends believe is a decent idea.
(Replies frozen) (Parent) (Thread) (Expand)
(no subject) - [info]immlass Expand
trialia From: [info]trialia Date: April 13th, 2009 06:26 pm (UTC) (Link)
I was pretty certain that the code he posted could do it, with a few modifications, but just as sure that he didn't, because of all the details he missed out. No screencaps = no proof, also.
(Replies frozen) (Thread)
bryant From: [info]bryant Date: April 13th, 2009 06:34 pm (UTC) (Link)
Screencaps are good, except it's too easy to snag some HTML, edit in whatever you want, and screencap that.
(Replies frozen) (Parent) (Thread) (Expand)
(no subject) - [info]trialia Expand
From: (Anonymous) Date: April 13th, 2009 06:37 pm (UTC) (Link)
What I can't get to jive with a troll theory are all the authors who as far back as Feburary got letters from Amazon saying this was a policy change.
(Replies frozen) (Thread)
bryant From: [info]bryant Date: April 13th, 2009 06:45 pm (UTC) (Link)
I wonder when it became large-scale? We know when we noticed it. It's easy to assume that it happened overnight. I don't know if there's evidence that it did.
(Replies frozen) (Parent) (Thread) (Expand)
(no subject) - [info]cookie_nut Expand