Is normal tap water okay to use in the pillow? August 06, 2019 14:29 Yes. If you use well water, you might want to add a few drops of bleach to the pillow water. Related articles Can the pillow be washed? How long can the water stay in my pillow before it needs to be changed? Which side of the pillow do I sleep on? How much water should be used? How do my patients fill the pillow?