Not if you're flying as a hobbyist. Under FAR 101.41
(a) The aircraft is flown strictly for hobby or recreational use;
(b) The aircraft is operated in accordance with a community-based set of safety guidelines and within the programming of a nationwide community-based organization;
(c) The aircraft is limited to not more than 55 pounds unless otherwise certified through a design, construction, inspection, flight test, and operational safety program administered by a community-based organization;
(d) The aircraft is operated in a manner that does not interfere with and gives way to any manned aircraft; and
(e) When flown within 5 miles of an airport, the operator of the aircraft provides the airport operator and the airport air traffic control tower (when an air traffic facility is located at the airport) with prior notice of the operation.
There is nothing in the FARs that specifically limits your altitude, however you could be considered a "hazard" if you enter controlled airspace without authorization. The 400' "recommendation" is meant to separate sUAV traffic from the lowest operating altitude airplanes and helicopter can potentially operate from.
As far as airspace authorization - read 101.41(e). These's nothing that says you could be denied operation. If this bill passes (and I believe it will) you need authorization to fly in class B, C, D and E airspace within an airport operation zone.