Adrian Ramos scored in the 45th minute, finally breaking through after Costa Rica's Randall Brenes was sent off in the 28th on a straight red card.
Colombia has three points in Group A, ahead of Bolivia and Argentina following their 1-1 draw on Friday in the tournament opener. Costa Rica, playing as a guest team in the South American championship, is without a point and is fielding a youth team, as is Mexico.
Two-time defending champion Brazil opens play on Sunday in Group B against Venezuela, and Paraguay faces Ecuador.
Short link: