The answer is yes, and you can play with them at MoMath. Specifically, the Twist and Roll exhibit has several convex shapes, which will roll arbitrarily far sideways when started at an angle that is not directly downhill.
http://1.bp.blogspot.com/-PTwGQo4rOjU/UM_aXhScXOI/AAAAAAAACX0/zZ_mCIxQLuk/s1600/M26.jpg
See also this video, which shows the motion of several of the objects.