Yvaine
Or, more to the point, since it sounds like it's already in, talk to someone you trust about getting an exorcism done. Don't rely on friends to do it, go to someone who's experienced with this sort of thing. If it's responding strongly to Christian religious items, you're probably best off asking a priest. You can go by yourself, or you can bring friends who've seen some of these things happen so that they can corroborate your story. There are still some exorcists left, they just don't talk about it a lot.
That and there is a major ordeal to get one done. It has to be brought to the church's attention, a bishop has to authorize an investigation, then it needs to be investigated. Then if it is deemed a case of possession a Bishop needs to approve of an exorcism. Then they need to find a qualified priest. The book "Hostage to the Devil" by Malachi Martin has a list of the requirements a priest must have before he can be given authorization to perform the exorcism. I'll say this, having read the list, there is a reason that Exorcism is not too common anymore. Not too many people in general meet the church's requirements.
Anyway after the authorization is given and a priest is chosen to do the Rite, You aren't even guaranteed success. Especially if there is any shade of Doubt in the mind/heart of the Exorcist.
It may be a route to take, but it is not an easy or pleasant one. Definitely stay around to help if you can as it is an interesting Ritual.
Has the Demon named itself yet? Have you tried to get it's name? Knowing its name will give a slight edge...not much, but it helps.