Typically, I wear a Small in J Crew sweaters, but I decided the X-small offered more of the fit I was looking for. The Small would have been fine if I'd wanted a more "relaxed" look. This sweater has a touch of "grandpa's sweaters" vibe...but in a good way...so I thought a slightly more fitted look would be better. Also, the length of the sleeves was better on the X-small. On the Small, I would have had to roll back the cuff. For reference, I'm 5'5", and the hem hits just under my rump.
Hope this helps....I'd love to hear from anyone else who's tried the Brynn!