That bit of wobble may be common but it is tunable.
What flight controller and what firmware are you using?
What size quad, what props and what battery?
Are you flying acro or a self level mode?
Have you got a couple of spare channels on your tx with at least one 3 position switch?
More information = more chance we will be able to help.