The X is not the X of algebra. The X is the Greek letter Chi, which is the first letter of the Greek word for Christ. I had a theology teacher who was forever abbreviating Christ as Xt, and that's what was going on with the Xmas abbreviation. But then the intent was mistaken to be "taking Christ out of Christmas".
The funny thing, as so often in English, is that even when we spell the word out, we don't say Christ Mass. What we say sounds more like Kris Miss.
Whether or not you are religious in any way,
I pray you: have a lovely Christmas Day.