Take your top 32 artists and pit them against each other in a knock-out style tournament. For each "match", randomly shuffle the artist's songs in iTunes or Spotify and play the first song. that comes up After you play both artist's songs, the artist with the song you preferred goes forward, and so on and so forth.
Some other rules about songs:
1. No song can be played twice
2. No live versions of a song to be played if there are album versions
Here's how my tournament went (artist play rank in brackets):
(1) LCD Soundsystem - Time to Get Away Beat (32) The Cardigans - Burning Down The House
(17) Fall Out Boy - Dance, Dance Beat (16) The Proclaimers - Letter from America
(25) Pendulum - The Island (Part 1) Beat The Hoosiers - Who Said Anything (About Falling In Love)?
(9) Passion Pit - Swimming In The Flood Beat (24) Lady Gaga - Bad Romance
(4) The All-American Rejects - Kids in the Street Beat (29) Beat Creedence Clearwater Revival - Born on the Bayou