Logo

dev-resources.site

for different kinds of informations.

Failing to get product information from Amazon with Perl

Published at
10/1/2021
Categories
perl
amazon
api
cpan
Author
davorg
Categories
4 categories in total
perl
open
amazon
open
api
open
cpan
open
Author
6 person written this
davorg
open
Failing to get product information from Amazon with Perl

I got a bit distracted this week. I was supposed to be writing a book, but as I was feeling a little unwell, I decided to do a little light recreational programming instead.

I'm an Amazon associate, which means if I share the right links I can drive people to the Amazon web site and get a cut of the money when they buy something. I started to muse about the idea of automatically generating pages that contain lots of the right links (books I recommend, films I enjoy, things like that). Ideally, I'd like to start with a list of ASINs (Amazon's unique product identifiers) and a TT template and end with an HTML page. I've done something like this before on the Line of Succession shop page.

But when I looked at the code that generates that page I realised that all the information about the products was hard-coded in a data file. I was copying the information from Amazon myself. I can't remember why I didn't grab the information from Amazon automatically.

Luckily, I've done this before. Ten years ago, I wrote a series of articles for Linux Format about Modern Perl. And the application I wrote over those articles was a reading tracker. And you added books to the system by giving it an ASIN which the program then used to get the title, author and cover image from Amazon. It did that using Net::Amazon. So I checked out the sample code, fired up the application and tried it out.

Only to find that it didn't work. The Amazon code was broken. Further investigation revealed that the API behind Net::Amazon had been turned off earlier this years and all requests to it returned the HTTP status "410 Gone". I found a conversation about this on the GitHub repo for the module, in which it's obvious that the author has little interest in updating the module to support the replacement API - which is unsurprising as it would mean a complete rewrite of the code.

I went off looking for alternatives. And found XML::Amazon. And, even better, there was a release of this module that happened after the API was turned off. But my happiness was short-lived when I discovered that it had exactly the same problem. I wondered how it had a new release if (presumably) all of the tests failed. It turns out that there's an environment variable that needs to be set in order for the live API tests to run. None of the CPAN testers will set that and I guess the author forgot to do that when releasing the new version.

(There's a potential blog post here about how the shrinking Perl community will see an increase of this kind of bit-rot on CPAN in the coming years - but this isn't that post.)

The replacement API for the one these older modules used is called the Product Advertising API and it's completely different to the older version. I guess Amazon have learned a lot about API design in the last few years - given that they seem to release dozens of new ones each month :-)

But I was enthused enough to start work on a replacement. It's called Amazon::API::ProductAdvertising. It's not yet ready for CPAN, but there's a GitHub repo. I was slightly scared by the signing mechanism that Amazon uses on API requests, but it turns out that Net::Amazon::Signature::V4 exists and makes that really easy.

So it basically works. I just need to work a bit on the parameter passing (there are a lot of possible parameters), add the documentation and package it up for CPAN. I'm hoping to get that done over the weekend. This blog post is mostly here to encourage me to do that!

cpan Article's
30 articles in total
Favicon
A link site of your very own
Favicon
App::cpx
Favicon
Combining calendars
Favicon
Installing CPAN modules from git
Favicon
Deploying Dancer Apps
Favicon
Feature release 1.36 of the Date::Holidays Perl distribution
Favicon
Building a tool to integrate Readwise.io highlights into my Zettelkasten via Perl
Favicon
Dist::Zilla::Plugin::GitHub::CreateRelease
Favicon
GitHub Organisations
Favicon
Mission (Almost) Accomplished
Favicon
Writing a CPAN module that talks to ChatGPT
Favicon
How to download cpanm in the portable way that supports Perl 5.8.1
Favicon
SPVM::IO 0.14 is released on Perl/CPAN
Favicon
Feature release 1.32 of Date::Holidays Perl distribution
Favicon
CPAN Module Tutorials
Favicon
cpanm - Installation of Perl Modules on User Directories 2022
Favicon
Wrapping a JSON API to access your personal data
Favicon
Highlighting members of the Perl family
Favicon
Create foo.a using Pure Perl and Archive::Ar
Favicon
Multiple ways to inheritance in Perl
Favicon
Taming the Moose: Classing up Perl attributes
Favicon
Some thoughts on Perl template processing
Favicon
Making Single-binary Release with pp
Favicon
The reports of Perl’s death have been greatly exaggerated
Favicon
Read The Fantastic Manual: How to get the most out of Perl documentation
Favicon
Failing to get product information from Amazon with Perl
Favicon
CPAN Release of TooMuchCode 0.18
Favicon
Fast Perl module installation with cpm
Favicon
CPAN installation as a test, with GitHub workflow
Favicon
CPAN Release of TooMuchCode 0.17

Featured ones: