Playing to win vs. playing to almost win
One of the big problems of D&D is that it’s a game built on the assumption that a) you have a continuous story, b) long term development, and c) constant hazards and combat that can remove characters from the story.
If you are constantly gambling the characters, at some point, you lose.
So the question is, what constitutes an appropriate challenge for your game? Does the DM play all-out, come what may? Do you play the monsters “realistically” to their tactical ability? Do you ramp them up or down based on the players’ ability? Are combats just for show and you fudge the dice whenever character death might come in?
See, these are very, very different games.
Old school D&D had no problem giving players big parties of NPCs, hirelings, or multiple PCs, so if you lost a character or two, you could still play.
And that players would use player knowledge to clear the dungeon- sure, your last character fell in the pit and died, but this character avoids it. This made it a lot like a videogame- the real contest is between the players and the level, not the characters.
Videogames also lack the requirement of a continuous story- if the character(s) die, you just go back to the last save, and as far as the plot is concerned, they never died.
Assuming you DO want a continuous story, and not having PCs magically learning everything their predecessors knew, you have two options. Either making character death impossible (by fudging, etc., at which point, combats are a show, and not really a challenge) or by decreasing the odds through tactics.
That is, you don’t try to win, you try to almost win.
Hitpoint Theory
At it’s most abstract, hitpoint based games are effectively a race- a race to reduce the opposition to 0 and reduce their damage output to 0 before they do the same to you.
All tactics, powers, abilities, and map movements are building blocks for setting up combinations to maximize your own damage output and minimize your opponents’ – and you also attempt to break their combinations at the same time.
The interesting thing is, what actually IS the most effective strategy and what APPEARS to be a threat, may not be the same thing in your players’ eyes. Sometimes the sub-optimal strategies work far better at giving the -feeling- of challenge than the actual best choices.
That said, let’s look at what the 3 main activities in tactical combat are:
Eliminating Units
A unit down is a unit not delivering damage. The optimal thing to do is to focus on putting down units by concentrating attacks on a single target, doing as much damage as possible, as quickly as possible, to the weakest available target – often either one that does a lot of damage or one that is capable of healing units.
While concentrating attacks is an optimal path for winning (in the broadest sense), it’s rarely the best thing to do in an encounter as the GM.
First, when a player sits out most of the action, helpless, it’s not fun for them. Second, usually only the player who is currently targeted really understands how nasty the concentrated damage is- everyone else often assumes they’ll be able to win the encounter, even if they lose one or two allies- forgetting that they’re losing on damage dealing and healing with each loss.
A better tactic is to concentrate fire on one target to seriously injure, but not down them, leaving the player fully aware of how bad it could get- and then switching targets. This often causes a lot of panic amongst the players while not being the most effective tactic to actually win.
Stealing Actions
Stealing Actions are anything that either completely stops a unit from acting for a round or more, or requires they waste an action fixing their condition(s). So stunning a target is a form of stealing actions, and so is throwing a net over them.
These are useful because they stop the target from doing damage, and probably from being able to heal damage- they get you ahead on the race to winning. Ideally, you want to lock down the opposition that does the most damage and/or heals damage – constantly.
This tends to slow combat, but is useful for raising player frustration and tension (overused, it locks players out of acting, and that’s no good).
Mitigating Abilities
Finally, if you can’t do direct damage or stop actions completely, you can add penalties or otherwise make certain actions less effective. This is stuff that reduces the abilities of the players to act effectively- whether it reduces damage outright (standing in a river vs. fire attacks) or in a more tactical sense (hiding in a narrow hall to avoid dealing with multiple opponents).
This is a great way to put pressure on the players to use secondary or alternate tactics and abilities to push them out of using the same old-same old methods.
Meeting the Players
All that said, you need to be clear with the players about what style of game you’re playing. Do you have a set difficulty and are going to play all-out? Will the players just have to rise to meet the occasion? Or will you set it to meet them and just challenge them at their level?
The hardest part is starting the game. A group of players can swing drastically in their ability, based on their individual strategic skills, their ability to coordinate, etc. An uncoordinated group is far less effective than a coordinated one.
Usually, I recommend starting with easy fights, which, you can ramp up the tactics as necessary. If you have 2 or 3 waves, or a vanguard and a reserve, you can play the first wave stupid, and improve the tactics with the remaining opposition. This is less about the actual fight itself and more about getting a feel for how good the players are going to work as a group.
Usually by the 3rd or 4th session, you’ll see what kind of teamwork you can expect. They’ve had enough time to familiarize themselves with the rules, their characters’ abilities and working with each other. At this point, any changes in player ability will be very long term.
Look for common strategies or tactics they fall into- simply creating encounters that push them out of being able to fall into that is challenging and interesting for the players- it forces them to have to actually think and not just go on autopilot.