Lem was the prime suspect of the murder of his newborn daughter, Raina, as he was the first to discover the infant dead of a broken neck in her crib. He knew that Mara Mattulich was the murderer, having seen her come out of the home shortly after the baby's death, but remained quiet, to ease the pain of his wife by making her think the baby died naturally.
However, Harra suspected Lem in murdering the baby, and sought her liege lord Count Vorkosigan for justice. Vorkosigan sent his son Miles to serve as his Voice and investigator. Lem initially hid from Miles, but eventually sought him out to prove his innocence via fast-penta, provided Miles not ask him who killed Raina. Miles assented, having figured out who killed Raina already.
Lem was acquitted, and advised by Miles that he and Harra go to Hassadar for Raina to undergo adult college education, with Lem financing the three years with his carpentry skills.
When Speaker Karal retired some years later, Lem became the Speaker of Silvy Vale, and began modernizing it, establishing public works such as a school named after Raina, a hospital, and a hydroelectric dam, much to Miles' approval and delight when he revisited Silvy Vale ten years after his murder investigation.
Lem and Harra eventually had two more children after Raina's death, a son and a daughter (said to be 4 and 1 years old respectively ten years after their sister's murder).