Webbots, Spiders, and Screen Scrapers - Michael Schrenk [83]
echo "\nReturned header:\n";
echo $header_array['FILE']."\n";
echo "Parsed server timestamp = ".$local_server_time_ts ."\n";
echo "Formatted server time = ".date("r", $local_server_time_ts)."\n";
Listing 19-1: A script that fetches and parses a server's time settings
When the script in Listing 19-1 is run, it displays a screen similar to the one in Figure 19-3. Here you can see that the script requests an HTTP header from a target server. It then parses the timestamp (which is identified by the line starting with Date:) from the header.
Figure 19-3. Result of running the script in Listing19-1
It is fairly safe to assume that the target webserver's clock is the same clock that is used to time the auctions. However, as a precaution, it is worthwhile to verify that the timestamp returned from the webserver correlates to the time displayed on the auction web pages.
Once the sniper parses the server's formatted timestamp, it converts it into a Unix timestamp, an integer that represents the number of seconds that have elapsed since January 1, 1970. The use of the Unix timestamp is important because in order to perform the countdown, the sniper needs to know how many seconds separate the current time from the scheduled end of the auction. If you have Unix timestamps for both events, it's simply a matter of subtracting the current server timestamp value from the end of auction timestamp. Failure to convert to Unix timestamps results in some difficult calendar math. For example, without Unix timestamps, you may need to subtract 10:20 PM, September 19 from 8:12 AM, September 20 to obtain the time remaining in an auction.
Time to Bid?
A sniper needs to make one bid, close to the auction's scheduled end but just before other bidders have time to respond to it. Therefore, you will want to make your bid a few seconds before the auction ends, but not so close to the end that the auction is over before the server has time to process your bid.
Submit Bid
Your sniper will submit bids in a manner similar to the other procurement bots, but since your bid is time sensitive, your sniper will need to anticipate how long it will take to complete the forms and get responses from the target server. You should expect to fine-tune this process on live auctions.
Evaluate Results
Evaluating the results of a sniping attempt is also similar to evaluating the purchase results of other procurement bots. The only difference is that, unlike other procurement bots, there is a possibility that you were outbid or that the sniper bid too late to win the item. For these reasons, you may want to include additional diagnostic information in the results, including the final price, and whether you were outbid or the auction ended before your bid was completed. This way, you can learn what may have gone wrong and correct problems that may reappear in future sniping attempts.
Testing Your Own Webbots and Snipers
The online store you used in Chapter 7 may also be used to test your trial procurement bots and snipers. You should feel free to make your mistakes here before you commit errors with a real procurement bot that discloses a competitive advantage or causes suspension of your privileges on an actual target website. Aspects of the test store that you may find particularly useful for testing your skills include the following:
The store requires that buyers register and authenticate themselves before making any purchase or bidding in any auction.
The prices in the store periodically change. Use this feature to design procurement bots that capitalize on unannounced price dips.
The address of the online test store is listed on this book's website, which is available at http://www.schrenk.com/nostarch/webbots.
Further Exploration
As a developer with the skills to write procurement bots, you should ask yourself what other types of purchasing agents you can write and what other parameters you can use to make purchasing decisions. Consider mapping out your particular ideas in a flowchart as I did in Figure 19-1 and Figure 19-2.
After