A U.S. District Court judge in Atlanta last week handed a five year prison sentence to Mark Vartanyan, a Russian hacker who helped develop and sell the once infamous and widespread Citadel banking trojan. This fact has been reported by countless media outlets, but far less well known is the fascinating backstory about how Vartanyan got caught.

For several years, Citadel ruled the malware scene for criminals engaged in stealing online banking passwords and emptying bank accounts. U.S. prosecutors say Citadel infected more than 11 million computers worldwide, causing financial losses of at least a half billion dollars.

Like most complex banking trojans, Citadel was marketed and sold in secluded, underground cybercrime markets. Often the most time-consuming and costly aspect of malware sales and development is helping customers with any tech support problems they may have in using the crimeware.

In light of that, one innovation that Citadel brought to the table was to crowdsource some of this support work, easing the burden on the malware’s developers and freeing them up to spend more time improving their creations and adding new features.

Citadel boasted an online tech support system for customers designed to let them file bug reports, suggest and vote on new features in upcoming malware versions, and track trouble tickets that could be worked on by the malware developers and fellow Citadel users alike. Citadel customers also could use the system to chat and compare notes with fellow users of the malware.

It was this very interactive nature of Citadel’s support infrastructure that FBI agents would ultimately use to locate and identify Vartanyan, who went by the nickname “Kolypto.” The nickname of the core seller of Citadel was “Aquabox,” and the FBI was keen to identify Aquabox and any programmers he’d hired to help develop Citadel.

In June 2012, FBI agents bought several licenses of Citadel from Aquabox, and soon the agents were suggesting tweaks to the malware that they could use to their advantage. Posing as an active user of the malware, FBI agents informed the Citadel developers that that they’d discovered a security vulnerability in the Web-based interface that Citadel customers used to keep track of and collect passwords from infected systems (see screenshot below).

Aquabox took the bait, and asked the FBI agents to upload a screen shot of the bug they’d found. As noted in this September 2015 story, the FBI agents uploaded the image to file-sharing giant Sendspace.com and then subpoenaed the logs from Sendspace to learn the Internet address of the user that later viewed and downloaded the file.

The IP address came back as the same one they had previously tied to Aquabox. The other address that accessed the file was in Ukraine and tied to Vartanyan. Prosecutors said Vartanyan’s address soon after was seen uploading to Sendspace a patched version of Citadel that supposedly fixed the vulnerability identified by the agents posing as Citadel users.

“In the period August 2012 to January 2013, there were in total 48 files uploaded from Marks IP to Sendspace,” reads a story in the Norwegian daily VG that KrebsOnSecurity had translated into English here (PDF). “Those files were downloaded by ‘Aquabox’ with 2 IPs (193.105.134.50 and 149.154.155.81).”

Investigators would learn that Vartanyan was a Russian citizen who’d grown up in Ukraine. At the time of his arrest, Mark was living in Norway, which later extradited him to the United States for prosecution. In March 2017, Vartanyan pleaded guilty to one count of computer fraud, and was sentenced on July 19 to five years in federal prison.

Another Citadel developer, Dimitry Belorossov (a.k.a. “Rainerfox”), was arrested and sentenced in 2015 to four years and six months in prison after pleading guilty to distributing Citadel.

Early in its heydey, some text strings were added to the Citadel Trojan which named Yours Truly as the real author of Citadel (see screenshot below). While I obviously had no involvement in writing the trojan, I have written a great deal about its core victims — mainly dozens of small businesses here in the United States who saw their bank accounts drained of hundreds of thousands or millions of dollars after a Citadel infection.

Following today’s breaking news about U.S. and international authorities taking down the competing Dark Web drug bazaars AlphaBay and Hansa Market, KrebsOnSecurity caught up with the Dutch investigators who took over Hansa on June 20, 2017. When U.S. authorities shuttered AlphaBay on July 5, police in The Netherlands saw a massive influx of AlphaBay refugees who were unwittingly fleeing directly into the arms of investigators. What follows are snippets from an exclusive interview with Petra Haandrikman, team leader of the Dutch police unit that infiltrated Hansa.

Vendors on both AlphaBay and Hansa sold a range of black market items — most especially controlled substances like heroin. According to the U.S. Justice Department, AlphaBay alone had some 40,000 vendors who marketed a quarter-million sales listings for illegal drugs to more than 200,000 customers. The DOJ said that as of earlier this year, AlphaBay had 238 vendors selling heroin. Another 122 vendors advertised Fentanyl, an extremely potent synthetic opioid that has been linked to countless overdoses and deaths.

In our interview, Haandrikman detailed the dual challenges of simultaneously dealing with the exodus of AlphaBay users to Hansa and keeping tabs on the giant increase in new illicit drug orders that were coming in daily as a result.

KrebsOnSecurity (K): Talk a bit about how your team was able to seize control over Hansa.

Haandrikman (H): When we knew the FBI was working on AlphaBay, we thought ‘What’s better than if they come to us?’ The FBI wanted [the AlphaBay takedown] to look like an exit scheme [where the proprietors of a dark web marketplace suddenly abscond with everyone’s money]. And we knew a lot of vendors on AlphaBay would probably come over to Hansa when AlphaBay was closed.

K: Where was Hansa physically based?

H: We knew the Hansa servers were in Lithuania, so we sent an MLAT (mutual legal assistance treaty) request Lithuania and requested if we could proceed with our planned actions in their country. They were very willing to help us in our investigations.

K: So you made a copy of the Hansa servers?

H: We gained physical access to the machines in Lithuania, and were able to set up some clustering between the [Hansa] database servers in Lithuania and servers we were running in our country. With that, we were able to get a real time copy of the Hansa database, and then copy over the Web site code itself.

K: Did you have to take Hansa offline for a while during this process?

H: No, it didn’t really go offline. We were able to create our own copy of the site that was running on servers in the Netherlands. So there were two copies of the site running simultaneously.

K: At a press conference on this effort at the U.S. Justice Department in Washington, D.C. today, Rob Wainwright, director of the European law enforcement organization Europol, detailed how the closure of AlphaBay caused a virtual stampede of former AlphaBay buyers and sellers taking their business to Hansa Market. Tell us more about what that influx was like, and how you handled it.

H: Yes, we called them “AlphaBay refugees.” It wasn’t the technical challenge that caused problems. Because this was a police operation, we wanted to keep up with the orders to see if there were any large amounts [of drugs] being ordered to one place, [so that] we could share information with our law enforcement partners internationally.

K: How exactly did you deal with that? Were you able to somehow slow down the orders coming in?

H: We just closed registration on Hansa for new users for a few days. So there was a temporary restriction for being able to register on the site, which slowed down the orders each day to make sure that we could cope with the orders that were coming in.

K: Did anything unexpected happen as a result?

H: Some people started selling their Hansa accounts on Reddit. I read somewhere that one Hansa user sold his account for $40. The funny part about that was that sale happened about five minutes before we re-opened registration. There was a lot of frustration from ex-AlphaBay users that weren’t allowed to register on the site. But we also got defended by the Hansa community on social media, who said it was a great decision by us to educate certain AlphaBay users on Hansa etiquette, which doesn’t allow the sale of things permitted on AlphaBay and other dark markets, such as child pornography and firearms.

K: You mentioned earlier that the FBI wanted AlphaBay users to think that the reason for the closure of that marketplace was that its operators and administrators had conducted an ‘exit scam’ where they ran off with all of the Bitcoin and virtual currency that vendors and buyers had stored in their marketplace wallets temporarily. Why do you think they wanted this to look like an exit scam?

H: The idea was to hit the dark markets even harder when they think the’re just moving to an other market and it turns to be law enforcement. Breaking the trust, so that [users] would not feel safe on a dark market.

K: It has been reported that just a few days ago the Hansa market administrators decided to ban the sale of Fentanyl. Were Dutch police involved in that at all?

H: It was a combination of things. One of the site’s employees or moderators started a discussion about this drug. We obviously also had our own opinion about it. It was a pretty good dialogue between us and the Hansa moderators to ban this from the site, and [that decision received] a lot of support from the community. But we didn’t instigate that discussion.

K: Have the Dutch police arrested anyone in connection with this investigation so far?

H: Yes, we identified several people in the Netherlands using the site, and there have already been several arrests made [tied to] Fentanyl.

K: Can you talk about whether your control over Hansa helped you identify users?

H: We did use some technical tricks to find out who people are, but we can’t go into that a lot because the investigation is still going on. But we did try to change the behavior [of some Hansa users] by asking for things that helped us to identify a lot of people and money.

K: What is your overall strategy in all of this?

H: Our strategy is that we want people to know that the Dark Web is not an anonymous place for criminals. Don’t think you can just buy or sell your drugs there without eventually getting caught by law enforcement. We want people to know you’re not safe on the Dark Web. Sooner or later we will come to get you.

Earlier this month, news broke that authorities had seized the Dark Web marketplace AlphaBay, an online black market that peddled everything from heroin to stolen identity and credit card data. But it wasn’t until today, when the U.S. Justice Department held a press conference to detail the AlphaBay takedown that the other shoe dropped: Police in The Netherlands for the past month have been operating Hansa Market, a competing Dark Web bazaar that enjoyed a massive influx of new customers immediately after the AlphaBay takedown.

U.S. Attorney General Jeff Sessions called the AlphaBay closure “the largest takedown in world history,” targeting some 40,000 vendors who marketed a quarter-million listings for illegal drugs to more than 200,000 customers.

“By far, most of this activity was in illegal drugs, pouring fuel on the fire of a national drug epidemic,” Sessions said. “As of earlier this year, 122 vendors advertised Fentanyl. 238 advertised heroin. We know of several Americans who were killed by drugs on AlphaBay.”

Andrew McCabe, acting director of the FBI, said AlphaBay was roughly 10 times the size of the Silk Road, a similar dark market that was shuttered in a global law enforcement sting in October 2013.

As impressive as those stats may be, the real coup in this law enforcement operation became evident when Rob Wainwright, director of the European law enforcement organization Europol, detailed how the closure of AlphaBay caused a virtual stampede of former AlphaBay buyers and sellers taking their business to Hansa Market, which had been quietly and completely taken over by Dutch police one month earlier — on June 20.

“What this meant…was that we could identify and disrupt the regular criminal activity that was happening on Hansa Market but also sweep up all of those new users that were displaced from AlphaBay and looking for a new trading plot form for their criminal activities,” Wainwright told the media at today’s press conference, which seemed more interested in asking Attorney General Sessions about a recent verbal thrashing from President Trump.

“In fact, they flocked to Hansa in droves,” Wainwright continued. “We recorded an eight times increase in the number of human users on Hansa immediately following the takedown of AlphaBay. Since the undercover operation to take over Hansa market by the Dutch Police, usernames and passwords of thousands of buyers and sellers of illicit commodities have been identified and are the subject of follow-up investigations by Europol and our partner agencies.”

On July 5, the same day that AlphaBay went offline, authorities in Thailand arrested Alexandre Cazes — a 25-year-old Canadian citizen living in Thailand — on suspicion of being the creator and administrator of AlphaBay. He was charged with racketeering, conspiracy to distribute narcotics, conspiracy to commit identity theft and money laundering, among other alleged crimes.

Law enforcement authorities in the US and abroad also seized millions of dollars worth of Bitcoin and other assets allegedly belonging to Cazes, including four Lamborghini cars and three properties.

However, law enforcement officials never got a chance to extradite Cazes to the United States to face trial. Cazes, who allegedly went by the nicknames “Alpha02” and “Admin,” reportedly committed suicide while still in custody in Thailand.

This story will be updated throughout the day. In the meantime, the Justice Department has released a redacted copy of the indictment against Cazes (PDF), as well as a forfeiture complaint (PDF).

Maybe some of you missed this amid all the breach news recently (I know I did), but Trump International Hotels Management LLC last week announced its third credit-card data breach in the past two years. I thought it might be useful to see these events plotted on a timeline, because it suggests that virtually anyone who used a credit card at a Trump property in the past two years likely has had their card data stolen and put on sale in the cybercrime underground as a result.

On May 2, 2017, KrebsOnSecurity broke the story that travel industry giant Sabre Corp. experienced a significant breach of its payment and customer data tied to bookings processed through a reservations system that serves more than 32,000 hotels and other lodging establishments. Last week, Trump International Hotels disclosed the SABRE breach impacted at least 13 Trump Hotel properties between August 2016 and March 2017. Trump Hotels said it was first notified of the breach on June 5.

According to Verizon‘s latest annual Data Breach Investigations Report (DBIR), malware attacks on point-of-sale systems used at front desk and hotel restaurant systems “are absolutely rampant” in the hospitality sector. Accommodation was the top industry for point-of-sale intrusions in this year’s data, with 87% of breaches within that pattern.

Other hotel chains that disclosed this past week getting hit in the Sabre breach include 11 Hard Rock properties (another chain hit by multiple card breach incidents); Four Seasons Hotels and Resorts; and at least two dozen Loews Hotels in the United States and Canada.

ANALYSIS/RANT

Given its abysmal record of failing to protect customer card data, you might think the hospitality industry would be anxious to assuage guests who may already be concerned that handing over their card at the hotel check-in desk also means consigning that card to cybercrooks (e.g. at underground carding shops like Trumps Dumps).

However, so far this year I’ve been hard-pressed to find any of the major hotel chains that accept more secure chip-based cards, which are designed to make card data stolen by point-of-sale malware and skimmers much more difficult to turn into counterfeit cards. I travel quite a bit — at least twice a month — and I have yet to experience a single U.S.-based hotel in the past year asking me to dip my chip-based card as opposed to swiping it.

True, chip cards alone aren’t going to solve the whole problem. Hotels and other merchants that implement the ability to process chip cards still need to ensure the data is encrypted at every step of the transaction (known as “point-to-point” or “end-to-end” encryption). Investing in technology like tokenization — which allows merchants to store a code that represents the customer’s card data instead of the card data itself — also can help companies become less of a target.

Maybe it wouldn’t be so irksome if those of us concerned about security or annoyed enough at getting our cards replaced three or four times a year due to fraud could stay at a major hotel chain in the United States and simply pay with cash. But alas, we’re talking about an industry that essentially requires customers to pay by credit card.

Well, at least I’ll continue to accrue reward points on my credit card that I can use toward future rounds of Russian roulette with the hotel’s credit card systems.

It’s bad enough that cities and states routinely levy huge taxes on lodging establishments (the idea being the tax is disproportionately paid by people who don’t vote or live in the area); now we have the industry-wide “carder tax” conveniently added to every stay.

What’s the carder tax you ask? It’s the sense of dread and the incredulous “really?” that wells up when one watches his chip card being swiped yet again at the check-out counter.

It’s the time wasted on the phone with your bank trying to sort out whether you really made all those fraudulent purchases, and then having to enter your new card number at all those sites and services where the old one was stored. It’s that awkward moment when the waiter says in front of your date or guests that your card has been declined.

If you’re brave enough to pay for everything with a debit card (bad idea), it may be the time you spend without access to cash while your bank sorts things out. It may be the aggravation of dealing with bounced checks as a result of the fraud.

I can recall a recent stay wherein right next to the credit card machine at the hotel’s front desk was a stack of various daily newspapers, one of which had a very visible headline warning of an ongoing credit card breach at the same hotel that was getting ready to swipe my card yet again (by the way, I’m still kicking myself for not snapping a selfie right then).

After I checked out of that particular hotel, I descended to the parking garage to retrieve a rental car. The garage displayed large signs everywhere warning customers that the property was not responsible for any damage or thefts that may be inflicted on vehicles parked there. I recall thinking at the time that this same hotel probably should have been required to display a similar sign over their credit card machines (actually, they all should).

“The privacy and protection of our guests’ information is a matter we take very seriously.” This is from boilerplate text found in both the Trump Hotels and Loews Hotel statements. It sounds nice. Too bad it’s all hogwash. Once again, the timeline above speaks far more about the hospitality industry’s attitudes on credit card security than any platitudes offered in these all-too-common breach notifications.

Further reading:

Banks: Card Breach at Trump Hotel Properties
Trump Hotel Collection Confirms Card Breach
Sources: Trump Hotels Breached Again
Trump Hotels Settles Over Data Breach: To Pay $50,000 for 70,000 Stolen Cards
Breach at Sabre Corp.’s Hospitality Unit

Axis Communications — a maker of high-end security cameras whose devices can be found in many high-security areas — recently patched a dangerous coding flaw in virtually all of its products that an attacker could use to remotely seize control over or crash the devices.

The problem wasn’t specific to Axis, which seems to have reacted far more quickly than competitors to quash the bug. Rather, the vulnerability resides in open-source, third-party computer code that has been used in countless products and technologies (including a great many security cameras), meaning it may be some time before most vulnerable vendors ship out a fix — and even longer before users install it.

At issue is a flaw in a bundle of reusable code (often called a “code library“) known as gSOAP, a widely-used toolkit that software or device makers can use so that their creations can talk to the Internet (or “parse XML” for my geek readers). By some estimates, there are hundreds — if not thousands — of security camera types and other so-called “Internet of Things”(IoT) devices that rely upon the vulnerable gSOAP code.

By exploiting the bug, an attacker could force a vulnerable device to run malicious code, block the owner from viewing any video footage, or crash the system. Basically, lots of stuff you don’t want your pricey security camera system to be doing.

Genivia, the company that maintains gSOAP, released an update on June 21, 2017 that fixes the flaw. In short order, Axis released a patch to plug the gSOAP hole in nearly 250 of its products.

Genivia chief executive Robert Van Engelen said his company has already reached out to all of its customers about the issue. He said a majority of customers use the gSOAP software to develop products, but that mostly these are client-side applications or non-server applications that are not affected by this software crash issue.

“It’s a crash, not an exploit as far as we know,” Van Engelen said. “I estimate that over 85% of the applications are unlikely to be affected by this crash issue.”

Still, there are almost certainly dozens of other companies that use the vulnerable gSOAP code library and haven’t (or won’t) issue updates to fix this flaw, says Stephen Ridley, chief technology officer and founder of Senrio — the security company that discovered and reported the bug. What’s more, because the vulnerable code is embedded within device firmware (the built-in software that powers hardware), there is no easy way for end users to tell if the firmware is affected without word one way or the other from the device maker.

“It is likely that tens of millions of products — software products and connected devices — are affected by this,” Ridley said.

“Genivia claims to have more than 1 million downloads of gSOAP (most likely developers), and IBM, Microsoft, Adobe and Xerox as customers,” the Senrio report reads. “On Sourceforge, gSOAP was downloaded more than 1,000 times in one week, and 30,000 times in 2017. Once gSOAP is downloaded and added to a company’s repository, it’s likely used many times for different product lines.”

Anyone familiar with the stories published on this blog over the past year knows that most IoT devices — security cameras in particular — do not have a stellar history of shipping in a default-secure state (heck, many of these devices are running versions of Linux that date back more than a decade). Left connected to the Internet in an insecure state, these devices can quickly be infected with IoT threats like Mirai, which enslave them for use in high-impact denial-of-service attacks designed to knock people and Web sites offline.

When I heard about this bug I pinged the folks over at IPVM, a trade publication that tracks the video surveillance industry. IPVM Business Analyst Brian Karas said the type of flaw (known as a buffer overflow) in this case doesn’t expose the vulnerable systems to IoT worms like Mirai, which can spread to devices that are running under factory-default usernames and passwords.

IPVM polled almost a dozen top security camera makers, and said only two (including Axis) responded that they used the vulnerable gSOAP library in their products. Another three said they hadn’t yet determined whether any of their products were potentially vulnerable.

“You probably wouldn’t be able to make a universal, Mirai-style exploit for this flaw because it lacks the elements of simplicity and reproduceability,” Karas said, noting that the exploit requires that an attacker be able to upload at least a 2 GB file to the Web interface for a vulnerable device.

“In my experience, I don’t think it’s that common for embedded systems to accept a 2-gigabyte file upload,” Karas said. “Every device is going to respond slightly differently, and it would probably take a lot of time to research each device and put together some kind of universal attack tool. Yes, people should be aware of this and patch if they can, but this is nowhere near as bad as [the threat from] Mirai.”

Karas said similar to most other cyber security vulnerabilities in network devices, restricting network access to the unit will greatly reduce the chance of exploit.

“Cameras utilizing a VMS (video management system) or recorder for remote access, instead of being directly connected to the internet, are essentially immune from remote attack (though it is possible for the VMS itself to have vulnerabilities),” IPVM wrote in an analysis of the gSOAP bug. In addition, changing the factory default settings (e.g., picking decent administrator passwords) and updating the firmware on the devices to the latest version may go a long way toward sidestepping any vulnerabilities.