The Ethics of Scraping

I’ve recently been given a commission to do a piece of coding involving grabbing information from another website and displaying it in a different manner. Obviously I won’t say much more about the brief – don’t want to annoy the customer or expose myself!

The brief got me wondering – how legal is it to grab content from another website? I guessed it was probably another one of the ‘grey areas’ that surround the internets at the moment, however another issue concerned me slightly more… how ethical was it? My script essentially steals information from a website for use in other ways, that the original publisher has no control over. Since the website in question publishes no open API or content feeds for this purpose, surely they mustn’t want this information being scraped from them? The website being scraped is essentially just a collection of many different items, published by their own individual owners. Do they want the information scraped and re-published?

I’ve heard of another case from a friend (again, won’t get into details) whereby a company is keeping a tight fist around its data. This is sort of understandable, but again the website in question is aggregating many different peoples data. Is the website being anti-competitive  by not releasing APIs?

I will sleep better once this script is out of development and off my server – I know I wouldn’t like it if someone else was doing this to my website! However – the customer is always right, and I won’t disappoint!

To kill a… Fiat Punto?!

Picture the scene, me driving happily up the M6 on Friday evening on my way to the Network Gathering in Windermere this weekend, when all of a sudden – disaster strikes!

*Dun dun dunnnnn*

Warning lights on the dashboard, engine starts to make funny noises, so I pull over into the Lancaster services. And then the car dies. Turn the key: nada. Its deceased… steam pouring from under the bonnet.

Some very kind gentlemen offers to lend me a hand pushing the car (I was blocking the slipway!) when to my embarrassment – the Highway Patrol pull up! Being pushed into a parking spot followed by a jeep with lots of flashing lights on… nice.

Anyway, 40 minutes later the AA pull up, the guy shoves some water into the engine, and off I toddle again. Make it about 2 miles down the road when it happens again! This time he decides its the head gasket that has gone, so off I go again, only this time being towed!

Arrive back in Poynton late on Friday night, and end up abandoning the car in the seediest part of Poynton Industrial Estate because the sodding garage is locked up for the weekend!

Cut to today, car goes into garage, I get told it’ll cost £550 to repaid. Plus the insurance is due up – another £1100. Bye bye all my money for uni for next term!