The Soay sheep of St. Kilda, a group of small Scottish islands, have been living free and unfettered for millennia. While these primitive sheep are still unmanaged, detailed records have been kept on them for the past 20 years. That makes St. Kilda a kind of natural evolutionary experiment, an island-scale Petri dish where sheep, not fruit flies, are the subjects.

Soays come in two colors: light brown and dark brown. The dark-brown sheep are bigger, and evolutionarily speaking, bigger should be better — big animals are more fit and more likely to survive. So over time, the frequency of dark-brown sheep in the population should be increasing. But the St. Kilda records show that the opposite is true.

Now a team of scientists led by Jon Slate, an evolutionary geneticist at the University of Sheffield in England, has discovered why. “It comes down to where on the genome the coat-color gene sits,” Dr. Slate said. “It just happens to sit next to this fitness gene.”

The researchers earlier identified the coat-color gene and discovered that dark was dominant. If a sheep gets the dark-brown variant from one or both parents, it will be dark brown. Only if it gets the light-color variant from both parents will it be light brown.

In the new research, reported in Science, they found nearby locations on the genome that affect body size, juvenile survival and reproductive success. They don’t know why these loci, as they are called, have the effects they do. But when the dark-color variant is passed to offspring, it’s accompanied by genes that positively affect body size but negatively affect juvenile survival and reproduction. When the light-colored variant is passed on, genes for smaller body size but better survival and reproduction come along for the ride.

That means light-brown sheep, and dark-brown sheep who have the dark-brown gene from only one parent, are more fit. Sheep with dark-brown genes from both parents are less fit, which accounts for their decreasing numbers. Their larger size is not enough to offset the other genetic factors. “If it wasn’t for the body-size gene, they’d be even worse off,” Dr. Slate said.