I did this once early on in my career and it paid off. I really wanted to work for a certain company in the field that related to my major in college (computer science and business administration). The job was a software engineer. I took a fairly significant pay cut, but made it up in time. It usually pays to take a long term view with your career, investing, etc.