Back With a Plan

I'm back. Fiji was awesome. Got a wee bit burnt... But no visible tan due to the copious amounts of sunblock I was applying. Damn. Also I left the sunblock at home so never found out about taking it through customs. Crystal clear water and fish everywhere. Also a lot of frogs. I found that strange. I don't have the camera with me at the moment so pictures will have to wait. Though they will be on here at some point.

As for tanks: No progress. Aside from the visuals there isn't too much to do (that is fun to code). So I'll have a chat with Foo to see where we are heading with this and will let you know.

In the mean time. I did come home to a large amount of Magic Cards at the door. I'm not expecting to use them all, large is an entire shoe box full which I got for NZD$10.50, so I going to try to make some of my money back by reselling them individually on TradeMe.

I'm not a hardcore fan so I have little to no idea of the worth of the cards, so what I am going to do is write another web crawler, like this one. Woo! Finally back to some coding. This is going to search StarCity for each card and return the value of said card for me, then dump it into a txt file; which I can directly import to a spreadsheet. Should let me know the most valuable to least.

I'm going to be using Python for this. Will post the source later on Ok? Ok.

Magic The Gathering

So I used to play Magic The Gathering when I was around 14. I have since found all out that a couple of my friends (including Foo) play. Sweet. It seems to be the case that each edition has better cards then the last. The newest set came out around 7 years ago... I think. So my cards are pretty average.

My play style: I have no bloody idea. Play shit and hope something awesome happens.

I'm quite good at puzzles and problem solving so strategy games seem to come naturally to me. Back in the day I made an adequate Myr deck. So what better thing to do then continue with the theme.

I'm not spending a lot of money on this. I'm a tight ass. So I found a couple stupid people on TradeMe, New Zealand's equivalent of eBay, that are selling ridiculous amounts of cards at $1 reserve. Managed to pick up 177 mostly white (not racist) cards in newish condition for $7.50. Not a bad deal. And a couple other decent deals.

Made a decent red/artifact deck that relies on the shit load of Myrs I have. Problem is that it takes a bit of mana to get going. Solution? Level-up creatures. I haven't got any of these yet...

Here's my current deck:

Lands 24
20x Mountain
4x Great Furnace (Note artifact)

Red Creatures 5
1x Dragon Mage Flying (He is awesome)
1x Goblin Machinist
2x Atog Sacrifice Artifact...
1x Megatog Sacrifice Artifact...

Red Spells 8
1x Shrapnel Blast Sacrifice Artifact...
2x Forge Armor (This card makes the deck)
2x Carbonize
1x Dragon Breath
1x Molten Rain
1x Erratic Explosion

Artifacts 4
1x Darksteel Pendant
1x Banshee's Blade Equipment
1x Myr Reservoir
1x Myr Matrix

Artifact Creatures 19
1x Myr Moonvessel Gives mana on death
2x Myr Enforcer Affinity for Artifacts
3x Iron Myr Gives mana on tap
2x Frogmite Affinity for Artifacts
1x Lodestone Myr
2x Cathodion Gives mana on death
2x Arcbound Crusher Modular
3x Myr Retriever
1x Myr Propagator
1x Yotian Soldier
1x Oxidda Golem Affinity for Mountains

Any comments?

Sorry this isn't software again... I'm posting at work again and haven't had time to update the blog at home.

miCrawler

Problem: At my work it was decided that we were to do mail drops. Rather than leaving the office and walking around the local area, we hit up local real estates offices etc and found all the addresses of houses for sale, our target audience. As it happened the real estate people were not helpful, there websites were. More helpful then these was the TradeMe website. As it turned out most of the real estate agents also listed the houses for sale on there anyway.
For a while it was my job to go through every listing and put them into a spreadsheet for some processing. After a day of this I thought it would be easier to write a web crawler to do it all for me.

tl;dr We need to get all the addresses off TradeMe, that are local and recent.

Solution:
Having recently done Python at university, I decided to use that. As all I wanted was a CLI program that I could implement quickly, it seemed to be a good fit. The program searches the list of listings for the listing number and then searches each listing for the address. If it has one, it is appended to a txt file. This file can then be added to a spreadsheet using imports.
Credit must go to a couple of places for this:
http://forums.devshed.com/python-programming-11/webcrawler-in-python-73318.html
http://www.daniweb.com/code/snippet216420.html
daniweb is a great source for help, although I can't stand the website. Found it tricky to navigate. Google helped with that.

Code:

#!C:/Program Files/Python/Python 2.7
import urllib, time
print "miCrawler version 0.1"
f = open('addressFile.txt', 'a')
# The below URL is the houses for sale in the manukau area, in list view, sorted by most recent, page 1
# May need to be updated occassionally.
tmManukau1 = 'http://www.trademe.co.nz/browse/property/regionlistings.aspx?mcat=0350-5748-3399-&v=List&key=100055&page='
tmManukau2 = '&sort_order=expiry_desc'
for i in range(0, 60):
    tmManukau = tmManukau1 + str(i) + tmManukau2
   
    page = urllib.urlopen(tmManukau).read()
    # Note that line 637 is the first occurance of a url for a particular listing.
    # The 9 digits following this string will represent a listing.
   
    auctionString = '/Trade-me-property/Residential-property/Houses-for-sale/auction-'
   
    pos = 1
    pos = page.find(auctionString)
    # Loops through this process until there are no more valid links on the page.
    while (pos != -1):
        # This will build a string for the URL of the listing found.
        listingURL = 'http://www.trademe.co.nz' + page[pos : pos + 77]
        listing = urllib.urlopen(listingURL).read()
        # Gets the address from the listing, the 26 skips the <td> and spaces.
        addSPos = listing.find('<td>', listing.find('Location:')) + 26
        addEPos = listing.find('</td>', addSPos)
        address = listing[addSPos : addEPos]
        try:
            int(address[0])
            address = address.replace('<br />', '\t')
            print (address + '\n')
            f.write(address + '\n')
        except ValueError:
            print (address + 'HAS NO NUMBER \n')
        finally:
            # Each address occurs twice, skip the second.
            pos = page.find(auctionString, page.find(auctionString, pos + 78) + 78)

Note:
This is very specific. It only searches Manukau and surrounding areas. Small changes on the urls will need to be made for that to change. The range in the loop determines which pages are to be read. Also I have no idea what that mcat number is and may need to be updated.
I suppose the main key here is this line:

 listing = urllib.urlopen(listingURL).read()

'listing' now contains the entire html code for the page. Performing some search functions on here for the auction numbers allows the program to find the relevent listing and from that page find the address. Simple enough.
The random hard coded numbers are there due to formatting. Easiest way I could think of to get around it. So that's that one. Never used all the 1,180 addresses. Kept the boss off my back for a while though.