So as per my previous post, I have written a very simple program to show me the value of Magic Cards. All you do is run the program and type in the name of the card. Then it prints out the value of the card for each version it is in. In addition it also saves and formats the printed bits in a txt file, which is compatable with spreadsheet software. See images below.
Simply type in the card you are looking for and press enter.
That is then saved here.
Which you can then import...
Here!
It also takes into account cards that do not exist. But not half typed. i.e. If I typed 'myr' it would return everything with myr in the title... No really worth fixing. More the error / aid of the website it's searching.
[Small error fixed: 'Mirrodin Besieged Singles' was picked up from the ads on the side bar. Now the first category is correct. I'm not replacing the screenshots though. You get the picture. (Spot the pun)]
Code below:
#!C:/Program Files/Python/Python 2.7
import urllib
print "miMTGCrawler version 0.1"
f = open('cardList.txt', 'a')
# The below URL is the link to a card. The card name is appended, also '+' replaces ' '. Case is irrelevent.
# e.g. 'Myr Enforcer' = 'Myr+Enforcer'
# Searches the website starcitygames.com for the information.
scLink = 'http://sales.starcitygames.com/search.php?substring='
while(True):
print "Please enter card name: "
card = raw_input()
if card == 'q':
break
#Builds the URL for the card.
cardLink = scLink + card.replace(' ', '+')
page = urllib.urlopen(cardLink).read()
# Search for the category. e.g. 'Mirridon'
categoryString = 'http://sales.starcitygames.com//category.php?cat='
# Jumps to the first listing.
pos = page.find('tooltip')
pos = page.find(categoryString, pos)
# Loops through this process until there are no more valid categories on the page.
while (pos != -1):
pos = page.find('>', pos) + 1
# This will build a string for the Card and it's categories and values of the listing found.
cardString = card + '\t' + page[pos : page.find('<', pos)] + '\t'
pos = page.find('">$', pos)
# Checks to see if a price was found following the category.
if pos == -1:
print 'There is no such card: ' + card
break
cardString = cardString + page[page.find('$', pos) : page.find('</td>', pos)]
print cardString + '\n'
f.write(cardString + '\n')
# Look for the next category.
pos = page.find(categoryString, pos)
Showing posts with label crawler. Show all posts
Showing posts with label crawler. Show all posts
Thursday, 17 February 2011
miMTGCrawler
Labels:
Code,
crawler,
link,
magic,
magic the gathering,
mi,
MTG,
python,
screenshot,
web,
webcrawler
Back With a Plan
I'm back. Fiji was awesome. Got a wee bit burnt... But no visible tan due to the copious amounts of sunblock I was applying. Damn. Also I left the sunblock at home so never found out about taking it through customs. Crystal clear water and fish everywhere. Also a lot of frogs. I found that strange. I don't have the camera with me at the moment so pictures will have to wait. Though they will be on here at some point.
As for tanks: No progress. Aside from the visuals there isn't too much to do (that is fun to code). So I'll have a chat with Foo to see where we are heading with this and will let you know.
In the mean time. I did come home to a large amount of Magic Cards at the door. I'm not expecting to use them all, large is an entire shoe box full which I got for NZD$10.50, so I going to try to make some of my money back by reselling them individually on TradeMe.
I'm not a hardcore fan so I have little to no idea of the worth of the cards, so what I am going to do is write another web crawler, like this one. Woo! Finally back to some coding. This is going to search StarCity for each card and return the value of said card for me, then dump it into a txt file; which I can directly import to a spreadsheet. Should let me know the most valuable to least.
I'm going to be using Python for this. Will post the source later on Ok? Ok.
As for tanks: No progress. Aside from the visuals there isn't too much to do (that is fun to code). So I'll have a chat with Foo to see where we are heading with this and will let you know.
In the mean time. I did come home to a large amount of Magic Cards at the door. I'm not expecting to use them all, large is an entire shoe box full which I got for NZD$10.50, so I going to try to make some of my money back by reselling them individually on TradeMe.
I'm not a hardcore fan so I have little to no idea of the worth of the cards, so what I am going to do is write another web crawler, like this one. Woo! Finally back to some coding. This is going to search StarCity for each card and return the value of said card for me, then dump it into a txt file; which I can directly import to a spreadsheet. Should let me know the most valuable to least.
I'm going to be using Python for this. Will post the source later on Ok? Ok.
Labels:
crawler,
fiji,
link,
magic,
magic the gathering,
MTG,
python,
tank,
tanks,
TradeMe,
web,
webcrawler
Subscribe to:
Posts (Atom)



