Smith proudly showed off the dashiki and kufi hat he wore to teach at the end of Ramadan two years ago.
Smith converted to the Islamic faith in college after being raised a Christian.
"Instead of more of a religious garment it really expresses African American culture," Smith said.
But the 9th grade Algebra teacher was told by the principal at Watson Chapel High, his attire was a distraction and he was made to go home and change.
After appealing that decision, the school board voted smith had the right to wear any religious garment he liked in a public institution.
"And that's when the harassment and hostility began," he said.
Smith filed a federal law suit claiming ever since he wore the attire to school he's been picked on, his work and lesson plans scrutinized.
As the Vice President of Watson Chapel's Teacher's Association, who tutors students to what he says are improved ACT scores, Smith believes the eyes looking over his shoulder are unwarranted.
"I've tried to be very instrumental in the process of the students," said Smith.
His federal lawsuit recently failed, but a county suit claiming he was punished by being made to work through lunch, is still underway.
Smith said his fight is about more than simply being re-paid.
"More importantly I want people to realize we have to begin to tolerate each other," he said. "If we want to grow as a society."
When reached for comment, the superintendent for the school district said when Smith was originally sent home the school's principal did not even know he was Muslim. Both declined an on-camera interview.