Thursday, May 31, 2018

New WiFi card testing - kismet_shootout

It's been a long time since I've officially tested wifi cards to see what is best and what you should skip.  Well, during the refit of the Wireless Capture The Flag kits, I've done some testing... and now I will share what I do, and what I found.

To start with, all of this is reasonably custom, and some of it I'm simply not sharing at this time.  Much of it, fortunately, is already fully open source, available to you, and already in Pentoo ;-)

This is going to be a multi part series, where I will introduce the tools, how they work, and some preliminary results.  Nothing on this page should be deemed a hardware recommendation, this was me setting up my test rig for the first time in a while.  The setup was physically entirely unfair, with wildly unmatched stock antennas and extremely close sources of noise.  Don't buy anything because you think it performed well here, and if you do, it was your idea not mine.

Part One:

To start with, I like to test a lot of cards:

PHY     Interface       Driver          Chipset
phy7    036ac           8812au          Realtek Semiconductor Corp. RTL8812AU 802.11a/b/g/n/ac 2T2R DB WLAN Adapter
phy11   036ach          8812au          Realtek Semiconductor Corp. RTL8812AU 802.11a/b/g/n/ac 2T2R DB WLAN Adapter
phy4    036acs          8812au          Realtek Semiconductor Corp.
phy12   036eac          8812au          Realtek Semiconductor Corp. RTL8812AU 802.11a/b/g/n/ac 2T2R DB WLAN Adapter
phy2    7822UAC         8812au          Edimax Technology Co., Ltd
phy1    AC56            8812au          ASUSTek Computer, Inc. USB-AC56 802.11a/b/g/n/ac [Realtek RTL8812AU]
phy6    awus1900        8814au          Realtek Semiconductor Corp. RTL8814AU 802.11a/b/g/n/ac
phy9    edup            8812au          Realtek Semiconductor Corp.
phy8    eub1200ac       8812au          Senao EUB1200AC AC1200 DB [Realtek RTL8812AU]
phy0    intel7265       iwlwifi         Intel Corporation Wireless 7265 (rev 61)
phy10   N600UBE         rt2800usb       Ralink Technology, Corp. RT3572
phy5    rlnknano        8812au          Realtek Semiconductor Corp. RTL8811AU 802.11a/b/g/n/ac WLAN Adapter
phy3    wdn4200         rt2800usb       Ralink Technology, Corp. RT3573


I really like to name the cards to make this easier (as you can see), this is done by adding a line for each card to /etc/udev/rules.d/99-nicnames.rules  Basically I plug each card in, and then bind it's mac to a useful name like this:

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:c0:ca:98:73:82", KERNEL=="wlan*", NAME="awus1900"

First, let's see if it monitors, and how well.  For this, I am currently using "kismet_shootout.rb", a ruby plugin for kismet which can be found in the path on Pentoo systems.  This tool is very simple, start kismet, and feed it all the wireless cards you want to test.  Then you start kismet_shootout.rb and tell it which cards to test, and on which channel.  For right now, the testing is fairly dumb, all the tool does it count packets seen, and whomever has the most packets is claimed to be "100%" and every other card then has a lower percent based on nothing but packet count.  This gives you a reasonable estimate of how many packets are coming in, but not if they are valid or not, and nothing is checking if the packets really are the same.  This tool only runs with "old" (stable) kismet right now, and that version of kismet doesn't have all the features the new kismet has, so this is as good as we have right now.  If someone were to reimplement the tool for new kismet, they could take advantage of the dedup mechanism to get hashes for each packet and see for real if the packets seen by all cards are the same.  Then the percentage could be based off the total number of unique packets and how many each card has seen, which would be slightly more useful.

INFO: Locking 036ac, 036ach, 036acs, 036eac, 7822UAC, AC56, awus1900, edup, eub1200ac, N600UBE, wdn4200 to channel 36
INFO: Waiting for sources to settle on channel...
INFO: Started at 2018-05-25 22:43:01 -0400
      Name   PPS  Packets Percent  Total  Elpsd
   N600UBE    30 40562918  93.50%
   wdn4200    59 38580847  88.93%
     036ac    37 42653025  98.32%
    036ach    35 43381613 100.00%
    036acs    29 37327675  86.04%
    036eac    33 39970296  92.14%
   7822UAC    33 40226936  92.73%
      AC56    33 37751328  87.02%
  awus1900    17 14268805  32.89%
      edup    27 35987645  82.96%
  eub1200a    28 36292940  83.66%
                                     361 118h3m
INFO: Locking 036ac, 036ach, 036acs, 036eac, 7822UAC, AC56, awus1900, edup, eub1200ac, N600UBE, rlnknano, wdn4200 to channel 44
INFO: Waiting for sources to settle on channel...
INFO: Started at 2018-05-24 16:25:55 -0400
      Name   PPS  Packets Percent  Total  Elpsd
  rlnknano     0   111123   5.83%
   wdn4200     0  1699403  89.19%
     036ac     0  1905478 100.00%
    036ach     4  1892517  99.32%
    036acs     0  1709680  89.72%
    036eac     1  1845425  96.85%
   7822UAC     2  1878870  98.60%
      AC56     1  1843208  96.73%
  awus1900     4   105650   5.54%
      edup     0  1615999  84.81%
  eub1200a     0  1686462  88.51%
   N600UBE     0  1864576  97.85%
                                      12 30h15m

As you can plainly see, I like to run tests a long time.  At least 4 days is best, because that makes me feel warm and fuzzy about the card not failing during an 8 hour hacking session.  Driver instability is a thing, so is kernel instability, and during just these monitor mode tests I had one card completely fail and get put in the bin.

As you can also plainly see, the Alfa AWUS1900 (the only r8814au tested) performs like crap, despite looking like a 4 antenna monster.  This is common with newer drivers and out of kernel modules (which this is both).  The hardware is probably great, but the driver is so bad that it's nearly unusable.  It's a bit hard to see due to the durations, but channel 44 is actually far more busy in my environment than 36, and the AWUS1900 was hot garbage on a busy channel, only doing slightly better on a very quiet channel.

The two best performers, Alfa 036ach and 036ac, traded spots a little big, but both maintained top performance on both 5GHz channels.  The surprising break out was the Alfa 036eac, which did remarkably well despite having no antenna, outperforming the Alfa 036acs which has an external antenna.

Again, this isn't a hardware recommendation, and this isn't even a completely ideal way to test things.  That said, I don't have an ideal way to test things, and so I'm doing what I can.  Now that I've got some idea of which cards don't suck, I will be redoing this testing (and posting the results for all to see) and start making some actual recommendations.  For now, enjoy kismet_shootout.rb, and if you want to reimplement it for new kismet you can find me both on kismet's discord as well as irc channel, and there will be glorious prizes for doing so :-)

-Zero_Chaos