Montezumas frontman Kristian Matsson started recording a set of rustic, gravelly-voiced tunes, ones that nodded to fellow Swedes Homesick Hank and Thomas Denver Jonsson, under the nom de solo act the Tallest Man on Earth in the early 2000s. His self-titled debut EP was released on Sweden's Gravitation Records in 2006. The "Pistol Dreams" single followed one year later, leading up to the release of the Tallest Man on Earth's first full-length album, Shallow Grave, which hit stores in 2008.