by Dichotomy and the Pros V Joes Crew


Upcoming Bsides for 2017

The following BSides are hosting a Pros V Joes CTF in 2017.

EventWhenPro regJoe RegVolunteer Reg
Bsides LVJuly 25-26 open Closed open
Bsides DETBD not open not open N/A

Player experiences

What's PVJ CTF like? Find out from our players from over the years!

Blue Team

A concise overview of the game.

A detailed perspective from a first time player.

A Blue Team Player's Guide

An enthusiastic report from yet another Joe

Red Cell

One Red Cell member's experience...

What a Rapid7 employee's thought of playing on the Red Cell


Into the crucible

The clock is ticking. You glance at the scoreboard and the service is still down. Why didn't that last change fix it? Suddenly a beacon alert pops up under your team name. Shit. Red Cell just took another device. You call out to the captain to let him know that you've lost another host. Frantically, you check the servers you were put in charge of, looking for some sign of pwnage. Netstat, process list...they're all clear. Oh, wait...what's that file in the web root directory? Better kill it just to be sure.

That's better, the beacon went away. But the service is still down, and the other team is ahead of you. Maybe by tweaking this config file here...that did it. It's green again. At least that stopped some of the bleeding.

Then someone taps you on the shoulder and asks if you could help them with their report, they just need to look at a spreadsheet that's on your fileserver. Who is this person? Were they on your team? You can't quite remember. You want to ask the captain, but he's busy with a couple of other team members, chasing out the other two beacons.

You're about to answer the stranger when suddenly your mail client dings. Another ticket has been opened by the scoring engine. Crap, we need to get some funky web app installed in the next thirty minutes or lose 1000 points. This guy is still asking for the spreadsheet, maybe it'd be okay to just let him have a peak so you can get that install done...

Shit, did another pwnage beacon just light up on the scoreboard?

Well, at least we've pulled ahead of the other team...for now...


What is Pros V Joes CTF?

This event is an opportunity for average users (Joes) to try their hand at both the offensive and defensive side of computer security. For the Pro's, it's a chance to hone and show off their skills, helping others to learn and better themselves. Joes are split up into teams, each with a Pro captain, and given their own network to defend against the other Pros in our Red Cell. Each team's network is full of servers and desktops running Windows, Linux, your standard services like DNS, Web, Mail, and others. Of course, there will be more than a few surprises...

For two days, players will attack and defend in live networks, breaking into each other's systems to steal flags for fame and glory. Each team is provided with their own network that is full of servers and workstations to defend. All of this gear is housed in a dedicated and isolated network that we affectionately call the Gaming Grid. Players need only bring a laptop to connect to the environment with. (Laptops will not be in the line of fire).


Scorebot

Dichotomy's Scorebot software is the heart of the ProsVJoes CTF. Designed from the ground up for the unique offense/defense style of play, it has grown along with the game and has served us well for many years now.

It's capabilities include (obviously) scoring the game, monitoring scored services for uptime, tracking flags and submissions, receiving beacons from assets compromised by Red Cell, and interacing with a ticketing system that the Blue Teams receive tasks from.

An open source project, we welcome pull requests with contributions. Right now, it's a little lacking in documentation, but we're working on fixing that. For those interested in trying it, we strongly recommend Gi0cann's Scorebot Vagrant project. It makes standing up an instance much easier than from scratch.


PvJ and SE CTFs have joined forces!

In 2015, the Pros V Joes CTF started working together with the Social Engineering CTF at Bsides LV. This brought a new and exciting dimension to the classical bits and bytes hacking of PvJ. Now, Pros and Joes alike also have to handle their physical security and their OPSEC. Social Engineering players have the challenge of trying to deal with a live organization of defenders that are trying to protect their information. Red Cell players of PvJ will have to choose a side (or not... ;)

Look for continued collaboration between SE and PVJ in the years ahead....


Other surprises...

The planning for PVJCTF at BSLV2016 is already well underway. In addition to continuing with this year's new features (SECTF, VoIP, and Graycell to name a few...) we have a few more surprises in the pipeline...

We might even have something to do with the Wireless CTF folks...

But more on that later. ;)


The Staff

Pros V Joes is the result of literally thousands of hours to build a realistic environment for players to attack and defend.

Gold Team

This is the admin team, the guys and girls that design, implement, and run the games. We've grown considerably at BSLV 2016, bringing the crew that helps build and run the games to seven people, all told.

RedCell

Pros V Joes now has a standing Redcell of 18 awesome, full-time, professional pen testers!

More player details coming soon...

Blue Team

To improve the game moving forward, we're working on recruiting permanent Blue Team Pros to assist us in helping to train the Joes. We now have five excellent folks, and are looking for more...

More player details coming soon...

Gray Team

In 2016 we've started a permanent crew of Gray Team Pros to help the volunteers at future games. Thanks to Mark and Guy for stepping up!

Other Contributors

Over the years, there have been a number of outstanding people who have donated their time, effort, blood, sweat, and tears to help build this CTF and make it what it is today, as well as what it will be tomorrow.

We'd like to give a shout out to PhobosJ, h4zm4t, and Sail0rl00n for their fantastic contributions


Prior year's BSidesLV CTFs

2016

Final Scores

team name

Infinite Improbability

The SYNdicate

Team Machine

JMP-ESP

Day 1 total

17,860

20,788

20,943

19,886

Day 2 total

3,114

20,148

-299,571

-24,335

Grand Total

20,974

40,936

-278,628

-4,449

Sponsors

We’d like to thank RSA for the donation of their appliance, without which the PCAPs from the 2016 Pros V Joes CTF would not be available.

Many thanks ot Maven Security for the donation of many hours of their employee's time to help prepare and run the 2016 BSLV Pros V Joes CTF. Without that contribution, the game would not have been nearly as successful!

Above all, we'd like to thank Wilmington University for the continued use of their facilities to host our CTF.

The Scoring Bug

This year, we had the misfortune to deal with a critical bug in Scorebot, the program that we use to run the ProsVJoes CTF.  This bug seriously impacted the scores displayed on the game board, which was inaccurately representing the state of the game for both days.

Right up front, I would like to personally apologize to all of our players for this flaw, and for the ill feelings this may have caused.  I have never seen a Pros V Joes CTF where so many people put in so much effort for so long before the actual game.  It is only natural that these fine people have an emotional investment in the game, and so may have been extremely disappointed in the final results once they were published on Twitter.  

Unfortunately, the bug is what is, and I can only do the best I can to make it right and fair after the fact, now that this year’s game has concluded.

As a small mercy, the bug was isolated in scope to only the ticket scoring portion of the game. The rules state that players must close tickets as a part of the game. To score them on this, the scoring software monitors the open and closed tickets. The design had been that the open tickets a Blue Team had would cost them 50 points per service scoring round (about 3 minutes, on average).

Here’s where the bug comes in.

Scorebot was docking each team 50 points per service round for every closed  ticket.

The bug was discovered on day two by one of our volunteers this year, Gambite, so a big shout out to him.

Unfortunately, no way exists to handle this bug during run time.  Once Scorebot starts, that’s it, she’s off and running for the duration. So our only choice was to fix things after game play stopped.

For the second day, we had the entire ticket database - every flag, when each was opened, and when each was closed.  So, we could go back and calculate what the final ticket score was for day two.

For the first day, we had no such advantage.  Historically, we wipe the ticket database at the start of day two, so that the prior day’s tickets are no longer counted, since each day’s game play is independent.   While scorebot generates a great deal of log data to track almost every event in the game, the ticket code does not currently log open / closed tickets for every scoring round.  That’s bug number two.   This was a simple, yet damaging oversight in the code base that will be corrected before the next game.

So, in the end, we were able to recalculate the correct scoring for day two, but we did not have the data needed to do the same for day one.

So what did the numbers look like for day two?  Here’s what they were, and what they changed to:

team name

Infinite

SYNdicate

Team Machine

JMP-ESP

Old Tickets

-198,000

-156,350

-63,650

-297,350

New Tickets

-4,250

-14,900

-314,750

-14,000

As can be seen, this is a drastic change.  Again, it is a consequence of the bug counting closed tickets as open when calculating the amount of points to take away each service round.

As a result, here’s what day two looks like after taking these new scores into account:

team name

Infinite

SYNdicate

Team Machine

JMP-ESP

services

20,464

20,648

19,779

20,165

flags

600

15,400

7,600

-3,500

tickets

-198,000

-156,350

-63,650

-297,350

beacons

-13,700

-1,000

-12,200

-27,000

Old Total

-190,636

-121,302

-48,471

-307,685

New Tickets

-4250

-14,900

-314,750

-14,000

New Ticket Total

3114

20,148

-299,571

-24,335

For day one, we did not have the data needed to make that sort of a recalculation.  As a result, our only options were to count the data as is, or discard that part of the score entirely.

We chose to discard the tickets, which seemed the most fair to the teams that had been diligent in closing their tickets.  Here is what day one looks like, with that choice:

team name

Infinite

SYNdicate

Team Machine

JMP-ESP

services

21,360

20,588

19,943

20,086

flags

400

1,400

2,100

2,600

tickets

-94,800

-100,950

-64,950

-104,750

beacons

-3,900

-1,200

-1,100

-2,800

total

-76,940

-80,162

-44,007

-84,864

total w/o tickets

17,860

20,788

20,943

19,886

So, adding up the adjusted first and second day scores as described above, we come up with the results that were published on Twitter:

team name

Infinite

SYNdicate

Team Machine

JMP-ESP

Day 1 total

17,860

20,788

20,943

19,886

Day 2 total

3,114

20,148

-299,571

-24,335

Grand Total

20,974

40,936

-278,628

-4,449

I have personally had multiple conversations with various Blue Pros and Blue Joes from multiple teams on this topic.  Thus far, everyone I have explained the circumstances to was understanding about the difficult choices that had to be made.  Indeed, three members from Team Machine, undeniably the group that was most impacted by this issue, have agreed to come on staff in spite of this snafu.

Regrettably, my team and I can do nothing about this issue beyond what the solution just described.  What we can do, however, is work hard to ensure our future games in the remainder of this year and into next do not suffer from this, nor from any other such egregious flaw.  To that end, we’re re-doubling our efforts to clean up the existing 2.x code train, while also restarting effort on the new 3.0 architecture.  We will be doing more code reviews and testing, working hard to do our best to prevent a recurrence.  Of course, mistakes will happen, but we intend to learn and improve from this one.

To that end, I am pleased to announce that Gambite, the volunteer who found the bug and helped me calculate the  correct scores, has signed on to our permanent staff as part of the Pros V Joes dev team.  


2015

The 2015 Pros V Joes CTF was our largest yet, with 44 players and 18 volunteers contributing to make the biggest CTF BSLV has ever seen! Everyone participating did remarkably well. Thanks all for playing, and I hope to see you next year at BSLV 2016!

Final scores

Team Score
Endtroducing 8,862
Labrynth Guardians 33,785
Castle Keep -20,781
Salty Goats 1,831

Sponsors

Many thanks ot Maven Security for the donation of many hours of their employee's time to help prepare and run the 2015 BSLV Pros V Joes CTF. Without that contribution, the game would not have been nearly as successful!

We'd like to thank Bijoti for the donation of their appliance, without which the pcaps gathered in 2015 would not have been possible!

Above all, we'd like to thank Wilmington University for the continued use of their facilities to host our CTF.

PCAPS

Full pcaps of the BSLV2015 PVJCTF will be posted shortly. Please stay tuned and watch this space...


2014

Sponsors

We'd like to thank Endace for the loan of their network traffic capture appliance, without which the posted pcaps would not have been possible!

Above all, we'd like to thank Wilmington University for the continued use of their facilities to host our CTF.

We were able to obtain exclusive pictures of what the game looked like through the eyes of Redcell...

PCAPS

PCAPs from the event are now available! Team Dentata Team Pequeninos


2013

Sponsors

We'd like to thank Wilmington University for the use of their facilities to host our CTF.

PCAPS

PCAPs from the event are now available!