Permanent -> Contract role

I was hired for a contract role at a US startup, then it got changed to permanent. It's been 4 months now. But the career here isn't enticing and I have another offer in hand. Can I tell the startup that I'm looking to move to another org and I can work in contract capacity until they find the new replacement?

Looking for suggestions on how to handle this and arrive at a middle ground?