As a sophomore at Syracuse University, Brown was the second leading rusher on the team. As a junior, he rushed for 666 yards (5.2 per carry). In his senior year, Brown was a unanimous first-team All-American. He finished 5th in the Heisman Trophy voting, and set school records for highest rush average (6.2) and most rushing touchdowns (6). He ran for 986 yards—third most in the country despite Syracuse playing only eight games—and scored 14 touchdowns. In the regular-season finale, a 61–7 rout of Colgate, he rushed for 197 yards, scored six touchdowns and kicked seven extra points for 43 points (another school record). Then in the Cotton Bowl, he rushed for 132 yards, scored three touchdowns and kicked three extra points. But a blocked extra point after Syracuse's third touchdown was the difference as TCU won 28–27.
Brown is a member of The Pigskin Club of Washington, D.C. National Intercollegiate All-American Football Players Honor Roll.
Perhaps more impressive was his success as a multi-sport athlete. In addition to his football accomplishments, he excelled in basketball, track, and especially lacrosse. As a sophomore, he was the second leading scorer for the basketball team (15 ppg), and earned a letter on the track team. His junior year, he averaged 11.3 points in basketball, and was named a second-team All-American in lacrosse. His senior year, he was named a first-team All-American in lacrosse (43 goals in 10 games to rank second in scoring nationally).
