You can remove contacts from a specific group if they are no longer matching the purpose of the group. The contact will stay inside your Contacts.
If a contact no longer matches a group, you can remove it from the group. The contact itself is not deleted, it is just no longer a member of the group in question.