Can you explain why we often see changes on commits on elements which we did not change, maybe just clicked on it to look for a color?
There're two kind of "invisible" changes in Sketch that you could see while synching your work via Versions:
Internal Sketch properties that don't necessarily affect the document layout but are still important to save and sync across your teammates for Sketch file consistency. Some of them could be ignored (and Versions already does ignore certain discardable changes) but some are vital.
Sketch internal file format migrations/updates. These are always invisible but without synching them you may end up with a broken Sketch file. Versions automatically sync everything in a safe manner so you don't need to worry about losing any changes.