"Most of the Spanish people are more intelligent than those people involved in racism," Diallo told BBC Sport on Tuesday.
"We believe it's only a very small group of people who want to spoil the football environment. The best answer for them is to play good football," he added.
The Elephants will take on Spain in a 2006 World Cup warm-up in Valladolid.
The Spanish Primera Liga campaign was marred by racial chants in different matches, the latest incident came at the expense of Barcelona striker Samuel Eto'o who wanted to leave the pitch in protest of the abuse by Real Zaragoza fans during their league clash on Saturday.
"At the moment we aren't thinking about walking off the pitch, we think people will understand that we only want to play football," Diallo stated.
"But if there is a problem on the pitch we will make a decision," he added.
Ivory Coast were drawn in a tough Group C in the finals in Germany alongside Serbia and Montenegro, Holland and Argentina.