Here is the quote I was thinking of. I couldn't find a copy of Teachings of the Prophet Joseph Smith so I couldn't verify whether it's in that book, but I found it in another source.
"He [Joseph Smith] also said many would awake on the morning of the resurrection sadly disappointed; for they, by transgression, would have neither wives nor children, for they surely would be taken from them and given to those who should prove themselves worthy. Again he said, a woman would have her choice; this was a privilege that could not be denied her." (Lucy Walker Kimball, as quoted in They Knew The Prophet, p. 136.)
Rock Is Dead. Long Live Scissors!
"The presentation or 'gift' of the Holy Ghost simply confers upon a
man the right to receive at any time, when he is worthy of it and
desires it, the power and light of truth of the Holy Ghost, although
he may often be left to his own spirit and judgment." --Joseph F.
Smith (manual, p. 69)