Interpreting “$X \text{ but } Y$“ as
$$X \wedge Y \wedge \diamond(X\wedge\neg Y)$$
is a reasonable starting point. (“X and Y and it would be possible to have X and not Y”.)

This works for the basic examples I found in online dictionaries:

- “He was poor but proud”
- “She’s 83 but she still goes swimming every day”
- “My brother went but I did not”
- “He stumbled but did not fall”
- “She fell but wasn’t hurt”

This correctly identifies that “he is a bachelor but unmarried” is not an appropriate use of “but”.

And this also shows the difference between such examples as:

- “That comment was harsh but fair.” (It was harsh and fair, while some comments are harsh and unfair.)
- “That comment was fair but harsh.” (It was fair and harsh, while some comments are fair and compassionate.)