Carmen grew up in Frankfurt am Main, Germany and in Lucerne, Switzerland. In Lucerne, he went to the Conservatory, but later studied bookkeeping. In 1975, he decided to turn to music. With bassist Mike Thompson, he founded the duo Carmen & Thompson, playing especially country music. The single "Time Moves On" reached #31 on the Italian charts in 1980, and in 1981 they appeared at the San-Remo-Festival performing the song "Follow Me".
After seven years of stage experience Phil Carmen founded the Picar Studios in Stein am Rhein, Switzerland in 1982 and recorded two solo albums which attracted little attention until 1985, when he had a hit with the LP Walkin' the Dog, which reached #3 in the Swiss charts. The single "On My Way in LA" was his greatest success and climbed in the summer of 1985 to #18 in Germany and #9 in Switzerland. "On My Way in LA" was also used in the television series Wild West in 1988. His 1986 album Wise Monkeys hit #1 on the Swiss charts, while the single "Moonshine Still" reached #10. His last successful album as a solo artist was City Walls (1987), which went to #10 in Switzerland. The corresponding promotional tour led to the Live in Montreux album with the same band line up of Brian Auger on organ, Pick Withers and Larry E. Van on drums, Dick Morrissey on sax, Steve Dawson on trumpet, Steve Evens on bass, and Sabine van Baaren, backing vocals.
In 1986 he founded the country band Clover Leaf. His 1993 album Skyline again featured Dick Morrissey and also Richard Tee. In 1996 he released his last album of cover versions of his favorite musician Bob Dylan.