I feel like Fitz only started liking Sophie out of envy. In Neverseen Keefe had shown his affection for Sophie more openly. This also might’ve made Fitz envious (jealous) of Keefe. This also might have made Fitz hate Keefe a little bit more in the next book.