I didn't speak up because flight attendants have difficult enough jobs without passengers constantly tattling on other passengers for indulging in creepy but legal behavior.

Also, a flight attendant's solution might well be to move to to a different seat, and the only available seats on that flight were middle seats.