It could be that one of the wires are not in the right order...this is a page with some info on identifying which coils pair up with what:
Other Stepper Motors
TBH, I have only fooled with stepper motors a few times I have had to use them in class, and I have never seen one with that many coils, but it looks like a stepper motor from your pic, and the behaviour you mention is what happens when the coils do not fire off in the right sequence.