<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:media="http://search.yahoo.com/mrss/"><channel><title><![CDATA[Lukas Pokorny]]></title><description><![CDATA[Thoughts, stories and file transfer.]]></description><link>http://lukas.pokorny.eu/en/</link><generator>Ghost 0.7</generator><lastBuildDate>Tue, 29 Dec 2020 03:57:33 GMT</lastBuildDate><atom:link href="http://lukas.pokorny.eu/en/rss/" rel="self" type="application/rss+xml"/><ttl>60</ttl><item><title><![CDATA[Introducting SFTP over WebSockets]]></title><description><![CDATA[SFTP and WebSockets are a perfect match. The combination of these two protocols eliminates the need for the black magic of AJAX uploads.]]></description><link>http://lukas.pokorny.eu/en/sftp-over-websockets/</link><guid isPermaLink="false">1d8b83bc-a992-4f0c-afc4-9fcd554cca67</guid><category><![CDATA[SFTP]]></category><category><![CDATA[WebSockets]]></category><category><![CDATA[Node.js]]></category><dc:creator><![CDATA[Lukas Pokorny]]></dc:creator><pubDate>Sat, 10 Jan 2015 20:46:53 GMT</pubDate><content:encoded><![CDATA[<p>Contrary to common belief, <a href="http://www.sftp.net/">SFTP</a> doesn't stand for <em>"Secure FTP"</em> or <em>"Secure File Transfer Protocol"</em>, but for <em>"SSH File Transfer Protocol"</em>. This is actually a double misnomer:</p>

<ol>
<li>Although SFTP is almost always layered on top of an SSH channel, it could just as well run over any reliable data stream.  </li>
<li>SFTP is not really a <em>file transfer protocol</em>, but rather a <a href="https://tools.ietf.org/html/draft-ietf-secsh-filexfer-02#section-1">simple remote file system protocol</a>.</li>
</ol>

<p>Because SFTP isn't related to the legacy FTP protocol at all, it doesn't suffer from it's shortcomings - FTP is notoriously firewall-unfriendly due to awkward distinction between control and data connection, and is not well-suited for much more that simple file upload or download.</p>

<p>Given the benefits of SFTP, it's not a surprise that lot of people have been looking for ways to use SFTP to transfer files from a web browser directly to an SFTP server. Some of them already had an SFTP server and wanted to eliminate the need of forcing their customers to install a stand-alone SFTP client, others just wanted to get rid of the nightmare of <a href="http://abandon.ie/notebook/simple-file-uploads-using-jquery-ajax">AJAX-based uploads</a>.</p>

<p><strong>The Past: Black Magic of AJAX</strong></p>

<p>At Rebex, we got numerous questions from those unfortunate people, and we always had to reply that unfortunately, it's impossible to implement a <a href="https://www.rebex.net/sample/sftp-asp-client/default.aspx">web-based SFTP client</a> unless the browser is extended with third-party plugins. And even though it's easily possible to transfer files between the <em>web server</em> and and SFTP server (using a suitable library), the transfer between the web browser and the web server has to be done by HTTP or the black magic of AJAX. Which was the very technology our customers wanted to get rid in most cases...</p>

<p><strong>The Future: SFTP over WebSockets</strong></p>

<p>Fortunately, those sad days will soon be over. Modern web browsers support <em>WebSockets</em>, a technology that makes it possible to open an efficient and persistent packet-based communication session between a web browser application and a web server. Because the SFTP protocol is packet-based as well, it's actually a perfect match. Additionally, the asynchronous nature of the SFTP protocol makes it ideal for the JavaScript environment. When the two protocols are combined, any modern web browser is perfectly capable of communicating <em>directly</em> with any SFTP server that supports <em>SFTP over WebSockets</em>, eliminating the need to use clunky and proprietary AJAX-based solutions.</p>

<p>Of course, there is a catch. SFTP servers only support SFTP over <em>SSH</em> <em>for now</em>, which means they can't be accessed by web browsers <em>directly</em>. I am currently trying to eliminate both of these issues:</p>

<ul>
<li>I'm writing a <a href="https://github.com/lukaaash/sftp-ws">SFTP over WebSockets server package for Node.js</a>. Includes a client as well.</li>
<li>I also plan to write a proxy that adds SFTP over WebSockets support to any server that currently supports SFTP over SSH.</li>
</ul>

<p>So far, everything is looking good. My <em>SFTP/WS</em> server is already working, and a proof-of-concept browser-based SFTP client should be ready soon!</p>

<p>If you are interested in SFTP over WebSockets, please <a href="http://lukas.pokorny.eu/#contact">let me know</a>.</p>]]></content:encoded></item><item><title><![CDATA[We have your email account]]></title><description><![CDATA[OSQA update checker sends all administrator emails and other statistics to DZone, which uses it to spam users with upgrade offers for AnswerHub. How can they expect me to trust them with my data now?]]></description><link>http://lukas.pokorny.eu/en/we-have-your-email-account/</link><guid isPermaLink="false">8997e0c4-3930-4c95-b690-9ac7440776d1</guid><category><![CDATA[customer relations]]></category><dc:creator><![CDATA[Lukas Pokorny]]></dc:creator><pubDate>Thu, 11 Dec 2014 16:11:00 GMT</pubDate><media:content url="http://lukas.pokorny.eu/en/content/images/2014/12/we-have-your-account.jpg" medium="image"/><content:encoded><![CDATA[<img src="http://lukas.pokorny.eu/en/content/images/2014/12/we-have-your-account.jpg" alt="We have your email account"><p><strong>TL;DR:</strong> OSQA's update checker silently sends all administrator emails and other <em>'statistics'</em> to DZone (its vendor) which uses it to spam its users with offers to upgrade to AnswerHub, a commercial alternative to OSQA. Do they seriously expect me to trust them with my data now?</p>

<p><em>I am well aware that some companies use questionable tactics. Usually, I don't care. But when they implement an 'updater' functionality whose main purpose is sending a list of administrator e-mails back home to the vendor (without my consent), it's simply too much.</em></p>

<p><em>Update:</em> Soon after I published this blog post, I got a reply and atteched it to this blog post with OSQA's permission. If only it arrived sooner...</p>

<p><strong>OSQA in troubles</strong></p>

<p>At Rebex, we have been using using <a href="http://www.osqa.net/">OSQA</a>, an open-source Q&amp;A system, to power <a href="http://forum.rebex.net/">our Q&amp;A forum</a> since 2010, when fee-based StackExchange 1.0 was discontinued. Although OSQA had its share of quirks, it mostly suited our needs. Unfortunately, a lot of OSQA sites have become a target of spam and the problem got quite out of hand recently, with hundreds of spam user accounts created every single day, possibly by some badly-written botnet. Dealing with this manually is no longer an option and we started looking into possible solutions.</p>

<p>One straightforward solution would be migrating to <a href="http://www.answerhub.com/">AnswerHub</a>, a hosted Q&amp;A platform from <a href="http://dzone.com/">DZone</a>, the makers of OSQA. Unfortunately, it has one major drawback - it doesn't have a publicly-known price, which usually translates to <em>it will cost you a lot</em>. But it was still on our shortlist, along with migrating <a href="http://www.question2answer.org/">Q2A</a>, which got harder because DZone removed the exporter modules from OSQA few years ago.</p>

<p><strong>We have your email account</strong></p>

<p>Fortunately, AnswerHub made our decision easier by their super-aggresive marketing tactics. When I received an e-mail from them offering a free trial, it annoyed me a bit because I don't remember sharing my e-mail address with them. When I received a similar e-mail three days later, it annoyed me even more. But at least I thought they are trying, I thought - my e-mail is not exactly private.</p>

<p>But when half of my colleagues received the same e-mail, some of them to their private e-mail addresses, we all became a bit suspicious. <em>Where did DZone got all those e-mails from?</em> When we discovered that these e-mails are exactly those we used to register ourselves on <em>our</em> forum, the suspicion grew even more. What is going on here? Have the folks at DZone planted some kind of backdoor into OSQA?</p>

<p>So we asked, and got a reply within minutes. Unfortunately, it was very strange:</p>

<blockquote>
  <p>"As the creators of OSQA we have your email account."</p>
</blockquote>

<p>Seriously? Now, I was becoming genuinely freaked out. The fact they used the term "email acount" instead of "email address" did not help either. I don't care about my e-mail address, but does this mean that the <em>creators of OSQA</em> might actually have e-mails of <em>our customers</em> who registered to our forum as well?</p>

<p>So I wrote another e-mail, where I asked:</p>

<blockquote>
  <p>"I’m a software engineer and I don’t understand! How is this possible? I have not installed your software myself and have not registered my e-mail with you. Does this mean your staff can actually access the administrator section of our forum?"</p>
</blockquote>

<p>I hit the "Send" button and impatiently waited for a reply. But there was none. I interpreted this as "there is indeed something fishy going on and we don't know what to say now" and started looking into OSQA source code for possible backdoors.</p>

<p><strong>Not a backdoor, but...</strong></p>

<p>Fortunately, I have not found a backdoor. Instead, I found <a href="https://github.com/dzone/osqa/commit/a529bba0262296c3057ada4f364616509f96af65">this</a>. The good folks at DZone added code to their software that sends e-mail addresses of all administrators <a href="https://updater.osqa.net/site_check/">back home</a> on every update check, and they somehow forgot to ask for a permission. And along with the e-mail list, they send some other <em>useful</em> info too:</p>

<p><img src="http://lukas.pokorny.eu/en/content/images/2014/12/osqa.png" alt="We have your email account"></p>

<p>They call this <em>statistics</em> in their code. And no, they don't ask for a permission to send this data - their UI clearly states that the purpose of the updater module is to <em>receive notifications about the latest updates</em>:</p>

<p><img src="http://lukas.pokorny.eu/en/content/images/2014/12/updates.png" alt="We have your email account"></p>

<p>No, I really don't see any mention of "sending a list of administrators and other sensitive information" back home! But at least our customer's e-mails are safe.</p>

<p>OSQA's commit history reveals that this started gradually. The first version of the updater module was <a href="https://github.com/dzone/osqa/blob/f6d6ed3eb1f778545c819fa93a7e664ed5c4e445/forum_modules/updater/views.py">almost innocent</a>. But soon they added <a href="https://github.com/dzone/osqa/commit/a529bba0262296c3057ada4f364616509f96af65">administrator email sending</a>, reporting of <a href="https://github.com/dzone/osqa/commit/9f3dfc73cd201e4deebfab89de3404c9059313d7">active user count since last update check</a>, and so on. In the end, a database of all those <em>statistics</em> became useful for their marketing department. And while I don't have anything against marketing, I really don't like companies stealing potentially sensitive personal and business information behind my back and without my consent. And no, the fact that the software was free does <em>not</em> help.</p>

<p>Although this might not be illegal, it's definitely unethical. How can I trust AnswerHub with my and customers' data now? It's simple - I can't! I have no idea what they might be doing with all those useful information behind our backs.</p>

<p>And by the way, I have not received any reply from DZone yet. It's been 24 hours already. Quite suprising for a company that proudly calls itself <a href="http://dzone.com/corporate/about">fast-paced</a>...</p>

<p><strong>Update 1:</strong> Just found out I'm not the <a href="https://twitter.com/ddbeck/status/542757083486371840">first one to complain</a>.</p>

<p><strong>Update 2:</strong>: Finally, I got a reply several hours after publishing this blog post:</p>

<blockquote>
  <p>Hi Lukas,</p>
  
  <p>I am sorry I didn't reply sooner but I had to sit down with my technical team to make sure I understood exactly how we got your email address, not account. However, it has come to my attention that you have already discovered this by your own research. We plan to update the installation documentation to make it clear that we collect this information when updating. AnswerHub never sells this OSQA data nor the data of any AnswerHub users, as AnswerHub customers own all their own data. </p>
  
  <p>I want to assure you that we truly meant nothing malicious by the email campaign. We've had some great success stories migrating OSQA users over to AnswerHub and our wish was just to make all OSQA users aware of the option. </p>
  
  <p>I am sure at this point you don't wish to have further communication or provide further information. If you wish to give us an address, I'd be happy to send you a package to further extend our sincerest apologies for the inconvenience. If not, I certainly understand.</p>
  
  <p>Sincerely,</p>
  
  <p>...</p>
</blockquote>

<p>Well, I have to admit the OSQA team can respond profesionally. When they do respond. Would they eventually respond as well had I not published the blog post? I don't know, of course. I hope they would.</p>]]></content:encoded></item></channel></rss>