Does it matter to deselect faces before going back to Object Mode?

When editing geometry, faces are automatically selected and can still be selected when going back to Object Mode. Does this have an effect on memory or performance? After completing geometry edits, if vertices, edges, and faces are still selected, it 'feels' weird, but I don't know if it really matters.