Normal to feel bad after ex arrested?

Depends on how good his lawyer is. You should be glad he's your ex and not your current boyfriend. It's kind of weird that you're more upset about the possibility of him going to jail rather than the fact that he burned a place down.