Flagging for closure is, for the purposes of the site, treated the same as an actual close vote in the sense that you cannot vote-to-close the same question twice; this effectively prevents close/reopen wars from dominating controversial questions, since participants can't just keep flagging/closing the same question whenever it's reopened, and giving people right on the 500-rep threshold the ability to both flag and close is just unfair to everyone else who can only do one or the other. So in that sense, the behaviour is very much by design.
The review queue does filter questions which have already been vote-to-closed by you, but it currently pays no attention to close-flags; this is probably a bug, and it has been brought up on MSO, but doesn't appear to have been resolved.
The only real option is to "Skip". Using the "Leave Open" option to remove the question from your queue would quite potentially remove the question from everyone's queue (it takes three "Leave Open" votes from the community to remove a question from the Close review queue), which is exactly the opposite of what you want.