The measure will take effect Oct. 1, AT&T said, and is intended to alleviate congestion on the network.
T-Mobile USA already throttles users who go over certain limits for data consumption.
AT&T stopped signing up new customers for "unlimited" plans last year. Instead, it now lets heavy users pay extra when they go over a certain data allotment.
Verizon Wireless also recently stopped signing up new customers for unlimited service.
AT&T says it will warn users when they are approaching joining the top 5 percent, and anyone subject to the speed limits will experience them until the next billing cycle starts.
The Dallas-based phone company says that what puts someone in the top 5 percent is usually streaming video or playing some online games.
AT&T won't count data use over Wi-Fi, just usage over the cellular network.