Ok, it's a little more complicated than it needs to be. You don't need a fuse AND a circuit breaker, but we'll overlook that for a moment. Let's look at the circuits in separate pieces so that we can be absolutely certain we don't get a "crossed wire".
BLACK WIRE - NEGATIVE
This guy never gets connected to red. Ever. Run the lead from the battery negative to the left side of the board in the photo like you have, run a black wire to the black wire on the LED light and forget it, black/negative is finished. No further connections on this side at all.
RED WIRE - POSITIVE
This is the side we generally switch and put fuses in. Your red wire goes from the battery to the right side of that board, which then passes power through a fuse and into the red wire attached to the middle, which runs off to the circuit breaker. If the circuit breaker is on (check it) it will pass power to that switch and if you have it wired like you do in the first photo, you have to flick the switch "off" instead of "on" to make the light come on. Change the lead that goes from the circuit breaker so that it doesn't join on the outer lug like you have it, but on the OTHER outer lug. Centre of the switch should go to the light.
Let us know how it goes.
Explanation of the switch: inside the switch is a 'rocker', it's like a see-saw. It balances on the middle connecting lug (so it's ALWAYS connected to the centre) and has a spring-loaded nib on a lever (with a central pivot) that pushes the rocker one way or the other. When you flick the switch off, the lever rotates around its pivot pushing the nib in the opposite direction, joining the other two connecting lugs. This means - holding the switch so that it flicks vertically up or down - when you push the switch down, the upper two connecting lugs are joined, and pushing the switch up connects the lower two lugs.