do I have to be a member of the church to come?

No!  Anyone - whether Christian or not - is most welcome to come to a church service.

does it cost anything to come to church?

There are no charges!  Church is free.  A collection is taken during Family Worship, but giving anything is entirely voluntary.  There is no obligation to give towards the collection.