I think it's problematic to lump Jack's mother and father together when you ask the "they" questions, because they are polar opposites when it comes to the issue of Jack's sexuality and Jack & Ennis's relationship.
Jack's mother is a kind, compassionate woman who accepted her son's sexuality and paid Ennis the respect afforded a grieving spouse because she appreciated that her son had been loved.
In contrast, Jack's father was a hateful man whose disdain for his son's sexuality and his relationship with Ennis was abundantly clear.
It's interesting to speculate (much information) about Jack's parents, and especially his father. I suspect it is more likely that Jack's father was "in between" the good, charitable version of his intentions (angry because Ennis couldn't make Jack happy) and the S.O.B. version, which would, if fully developed, have certainly removed or destroyed the shirt years ago and would probably not even speak to Ennis. (Maybe spat at him instead of in front of him.) Jack's (probably long-suffering mother) is less mysterious, but no less heart-wrenching.
I think Father Twist was probably sad and disappointed with his son, his personal condition, and the condition of his farm, which was probably the (pathetic) culmination of his life's work. Everything seems to have disappointed him. I didn't get the feeling that he would have kicked Jack and Ennis out if they decided to help him, so probably the gay thing was less tragic than the deteriorating farm thing.
I think that Jack's parents, Ennis, and everybody else in the film that loved Jack (and probably most of us!) loved Jack as best they could, even though it probably wasn't enough. This is one of the central themes of the film and story.