Gausha came out on the wrong end of a 16-15 decision after an exciting fight with Indian Vijender, while earlier the 19-year-old Californian Ramirez was a 15-11 loser to Fazliddin Gaibnazarov of Uzbekistan.
The two losses leave the United States with just Errol Spence and Rau'shee Warren remaining from a start-list of nine men, as well as their three women fighters who begin their respective campaigns at the weekend.
Big-punching Gausha threw in a strong final round but could not sway the judges to swing the bout in his favour, while Ramirez said he had no complaints over the nature of his defeat to his Uzbek opponent.
Ramirez said: 'I came out trying to be patient but I guess being patient didn't work for me tonight. I have no complaints about the decision. It has been great to be here as part of the team and it has given me a lot of opportunities.'
Earlier, Vasyl Lomachenko showed the skills that have seen him routinely highlighted as the best amateur boxer in the world as he got his quest for a second Olympic title under way with victory over the Dominican Republic's Wellington Romero.
The Dominican fighter was simply dazzled by the sharp, accurate shots of the 24-year-old Ukrainian Light Weight, who turned a 9-1 advantage at the end of the first into a comfortable 15-3 victory at the end of three one-sided rounds.
Lomachenko is making his Olympic debut at Light Weight having won the Feather Weight title in Beijing, where he accumulated 58 points in his five bouts and conceded only 13, also winning the Val Barker Trophy for the best boxer of the tournament.
Lomachenko, also a two-time world champion, will next face Puerto Rican teenager Felix Verdejo Sanchez, who beat Tunisia's Ahmed Mejri 16-7, with at least a bronze medal guaranteed for the winner.
Cuba's fourth seed Yasnier Toledo Lopez did not have it all his own way against China's world number nine Liu Qiang, but produced the sharper work in a rather mundane affair to take the verdict 14-10.
Meanwhile, Great Britain's Anthony Ogogo produced arguably the greatest performance of his career to defeat Ukraine's reigning world champion Ievgen Khytrov via a nail-biting judges' decision to move within one more win of a Boxing medal.