I used a chemical called Goof Off. You can buy it at Walmart or any hardware store. Just apply a little bit to a cotton ball and it wipes right off. A lot of pony customizers use this stuff to get rid of the cutie marks.
I’m sure that acetone will also work, but I like Goof Off, because it’s a little safer on the vinyl.