University unions that represent both faculty and administrators pretty quickly get owned by the administrators, who outnumber us substantially. Administrators can take annual leave. I'm not sure that annual leave ever really exists for academics. It's pretty much impossible not to be turning over problems in your head, constantly, whether you're at the desk or whether you're sitting pool-side. Unless you're in the bench sciences where you're tied to a lab bench, you can work from anywhere that has a reasonable internet connection - and even that can be optional for some kinds of work.
Tenure and promotion are tournament games. If you take your allowed annual leave seriously and somebody else doesn't, you're down a rung for tenure or promotion. If there's downsizing pressure, taking leave puts you a rung closer to the chopping block and a step farther away from being able to get hired elsewhere.
The bargain seems to be that annual leave doesn't really exist* but nobody really takes attendance either. So if you need to stay home with a sick kid or whatever, it's pretty easy (so long as you're not on deck to lecture that day). A couple weeks on the beach as proper leave? Well, you can do it. As much as you want. It's totally up to you.
* ...although we all have to fill in forms claiming that we took 5 weeks' leave and indicating the days nominally taken so that untaken leave can't accrue on the books as a liability. The strong hint is that anybody making a fuss about it will cause the whole place to start putting in time clocks.
