By all means, don’t go against conscience.
I had to do a little research, but it seems fairly clear to me that totem poles actually function more like iconography than like idols. They don’t seem to “embody” the god/spirit the way an idol does, but they can refer to them pictorially.
But that said, I can clearly tell the difference between, for example, a Hindu idol and a Hindu icon, but I would never keep either one in my house.