If Baerlon is taxing and getting taxed, Camelyn is getting her cut from the leaf. There is also ore near Baerlon, and I believe gold. the Crown certainly wants a piece of that. Although, in relation to Camelyn, if you're in Baerlon, you're practically in the Two Rivers.

 

I don't think the terrain favors bypassing Baerlon by much. It's surrounded by two rivers, hills, and a lake with mountains behind it.I don't know who guards Baerlon, or how far they patrol.

Morgase's thoughts about this:

 

"Something about the Two Rivers, some spark she could not quite fan to life, tugged at her. The region was hardly part of Andor at all, and had not been for generations. She and the last three queens before her had been hard pressed to maintain a modicum of control over the miners and smelters in the Mountains of Mist, and even that modicum would have been lost had there been any way to get the metals out save through the rest of Andor. A choice between holding the mines’ gold and iron and other metals and keeping the Two Rivers’ wool and tabac had not been difficult."

Min-LOC The First Message:

 

"I was born in the Mountains of Mist, Elayne, at the mines. Your mothers writ runs pretty thin, that far west."

I can understand why they wouldnt send people to the TR for taxes. If you were monarch would you risk losing land to your south or the river to your east to tax at most a thousand farmers? The best solution is to tax the tobacco when it enters your more populated areas.

Morgasse said they always had problems with the contentious miners from the Mountains who basically kept the region somewhat segregated. It's yours on the map, you get the minerals and just leave the regular folks alone. That's how she seemed to think about it.