"A Tropical Storm Warning has been issued from Cape May, New Jersey northward to Watch Hill, Rhode Island, including Long Island and Long Island Sound," the National Hurricane Center said.
The warning means that tropical storm conditions are expected somewhere within the warning area within 36 hours.
Forecast models show Fay moving inland over the northeast United States on Saturday.
Once the storm moves over land, it should weaken quickly.
"Fay is expected to produce 3 to 5 inches of rain along and near the track of Fay across the mid-Atlantic states into southeast New York and southern New England," the National Hurricane Center said.
Other impacts include winds of 40-50 mph along the coast, coastal flooding and beach erosion.
"Residents along the mid-Atlantic and Northeast coasts should expect conditions similar to a Nor'easter," according to Ward.