When working with iTerm or a console in general I hate it that I can't jump words forward or backwards with ALT + Arrow keys. Now I found a way to do this.
And it's so simple I just didn't know the term "Natural Text Editing". Which is what you have to choose in the preferences.
Preferences > Profiles > Keys > Load Preset > Natural Text Editing