In Tbilisi the tap water is safe to drink however, outside Tbilisi I would be inclined to drink bottled water which is widely available. If your visit is short I would recommend sticking to bottled water as different people’s stomachs can take time to get used to the water although it’s fine for brushing teeth, etc. Bottled water in glass and plastic bottles is widely available.