The gym I use has arc trainer machines insted of elliptical machines. They are very similar, except on an elliptical your feet make a circular motion and on the arc it's a linear back and forth more like cross country skiing. I have some joint issues myself, and I have to stick with zero impact exercises. The arc trainer I can do without pain, it's awesome. Just make sure you use one that has sturdy handgrips to help with your balance.
Water aerobics would be a good option, unfortunately there aren't any classes near me. Typically you don't have to be able to swim. Beginner's yoga might be an option for you. I am not flexible in the least, but yoga helps me tremendously with my mobility. I have one titled "AM and PM yoga for weight loss" that ANYBODY can do, and I have seen yoga dvd's targeted for people with back issues. There's also a PBS series titled "Sit and be fit" that is done from a chair.
If I keep starting over, eventually it will stick, right?
Current weight: 140
Goal weight: 135