I mostly agree with you. I have finished the university, and looking at how useless most of the things I have learned there are, I have some regrets for the time I have wasted.
But there are two things I'd like to comment to your post, Tynan. First of all, I believe most people do not have the self discipline necessary to self educate - especially when they are 18 years old. As Arun says, higher degree studies help instill a sense of discipline, which I think most people lack naturally. For me, what you are saying might work now, when I am 28 years old. Would it have worked when I was 18? I doubt it, looking back I was completely clueless to how the world worked and what I need to do to succeed. And looking to younger people around me, I see the same lack of focus and purpose.
The second thing is - colleges have been around for hundreds of years, in some form. What you are outlining in your post has only become possible in the last 10 years or so. You say learn poker - easy to do now all by yourself, with all this information at your fingertips. Harder to do even 20 years ago, unless you ran across some mentor to school you. The same goes for learning languages. And travelling has become increasingly affordable only in the last decades also.
There will be changes in the way formal education works, but you have to give it some time. The future belongs to projects like khan academy, but it will take time to replace an age old system.
By the way, also check out Ken Robinson's talk about how education is broken. Or his talk on TED about how schools kill creativity.