Rooter invades Russia 
There's an amazing new SCIgen story out of Russia. Below is the full story, in the words of the mastermind himself, Mikhail Gelfand. But first, the executive summary:

  • The original Rooter paper, translated into Russian, was accepted into a nationally accredited journal.
  • The paper received mostly positive reviews.
  • After the revelation that the paper was fake, the story became a national news sensation. Mikhail even appeared on radio and TV shows.
  • The Russian word for "rooter" ("Korchevatel", a kind of machine that digs up roots) became synonymous with nonsense and low-quality science.


Now enjoy the story of Korchevatel, as written by Mikhail Gelfand:




To understand it, you need some background. In Russia all Ph.D. (Candidate of Science) and D.Sc. (Doctor of Science, the higher one) degrees, irrespective of an awarding university or institute, have to be validated by the state body called Higher Attestation Commission (HAC; in Russian it is "Vysshaya Attestacionnaya Kommissiya", VAK, or, in Cyrillic, BAK). Several years ago, in an attempt to improve the quality of the defended theses, BAK published a list of "approved" journals, in which the main results of a dissertation had to be published. Other publications sort of do not count. However, this list was immediately infiltrated by inferior journals with no or only superficial peer review, offering fast publication for money. "Journal of Scientific Publications of Aspirants (Ph.D. students) and Doctorants (those working towards the D.Sc. degree)" (JSPAD) is one of them. It was registered in 2006 and carried out an aggressive advertisement campaign in the form of bogus posts at internet forums frequented by students. The posts were written as if by an aspirant who had published in the journal and liked it, but in fact they were posted by the chief editor himself (as seen from the limited set and overall similarity of the nicknames used).



As a result, the journal published about 300 papers in the first year of its existence and about 500 in the second year. At some point it has attracted our attention. By "us" I mean a group of scientists and scientific journalists publishing a young biweekly newspaper "Troitsky Variant" (http://www.scientific.ru/trv/2008/). Firstly, we analyzed the papers published in JSPAD and found a lot of crazy junk. Secondly, we researched the chief editor (internet is a wonderful resource) and found him to be a small-scale business lawyer with no scientific degree from Kursk, a medium-sized region center in the South-West Russia (best known for the largest tank battle of the Second World War). And thirdly, we decided to test the journal peer review procedure using SCIgen.



I took your original Rooter paper and asked colleagues in my institute (A.A.Kharkevich Institute for Information Transmission Problems) to translate it into Russian using their automatic translation program ETAP-3. Then I did minor editing to get rid of usual problems of an automated translation and played with the reference list and acknowledgements. In the references I added some invented papers with funny Russian titles (assuming that a Russian paper with absolutely no Russian references might look strange) and, to be fair with the journal and level the field (skewed after editing the translation), substituted one of the English author names for Softporn. For the same reason, in the Acknowledgements, I placed "thanks to Professor Gelfand who introduced the author to the problem of publication of random texts". The paper was signed by an invented Ph.D. student from a non-existing Moscow institute. In that form the paper was sent to the journal.



The timetable was as follows:

August 6: submission by e-mail.

August 7: notification from the journal and request to pay (NB: before review; I wonder how they were planning to return the money in case of rejection :-)

August 8: scanned copy of the bank slip sent to the journal.

August 11: notification that the paper is sent for the review.

August 13: reviewer's comments (consisting of a number of standard scored points, see below, and highlighting in the original file of sentences requiring editorial changes).

August 15: revised paper e-mailed to the journal.

August 15: notification of acceptance.

September 2: notification that the hard copy of the journal is sent by mail.

September 10: paper appeared at the journal internet site.



The scores were:

actuality: high
the choice of the study subject: correct
setting aims: logical
novelty: excellent
depth: sufficient
structure: good
value of methods: excellent
style: non-satisfactory
practical efficacy: excellent
coverage of literature: excellent



After that, we've published the entire story, together with selected sections of the paper and some additional material (including the results of our research on the chief editor, the Editorial Board of JSPAD, and historical notes about some classical papers turned out to be jokes) in "Troitsky variant" (http://www.scientific.ru/trv/2008/#13). The story has been picked up by a couple of news sites, and then attracted a lot of attention. In particular, it was featured by two of three main TV channels (e.g. http://news.ntv.ru/141780/ with me and the chief editor of JSPAD) and also radio (including "Radio Liberty" that made a mini-series of three interviews with me http://realaudio.rferl.org/RU/2008/10/1 ... U-clip.mp3, the chief editor http://realaudio.rferl.org/RU/2008/10/1 ... U-clip.mp3, and the head of BAK http://realaudio.rferl.org/RU/2008/10/1 ... U-clip.mp3). The story was also featured in a number of newspapers and weeklies, including the Russian version of Newsweek that published a short story with your (not mine, for a change) photograph. The JSPAD editor blamed everything on "Moscow journal mafia" afraid of competition and an unnamed Moscow institute that by an agreement with the journal was reviewing all JSPAD papers (a lie: firstly, the peer-review system does no work like that even in Russia, secondly, no institute is capable of reviewing papers in all sciences from math and physics to biology and engineering to history, sociology and political science, and thirdly, in any case proper reviewing requires more than two days). Needless to say, the paper was promptly removed from the journal site. Still, I have the hard copy of the September issue and backups of the electronic version of the paper are abundant in the Russian internet.



The administrative consequences were rather fast. In two weeks the journal was kicked out from the BAK list. More importantly, at the same time BAK announced requirements for journals willing to be listed (effective Sep. 1st, 2009), which are mainly reasonable and, hopefully, would preclude similar journals from getting in the list. Or, at least, make it more more difficult for them. The editorial board of the journal resigned (at least, the respective link on the journal main page disappeared).



After a while, the story was mentioned in the Russian Wikipedia (http://ru.wikipedia.org/wiki/%D0%9A%D0% ... 0%BB%D1%8C) and the Encyclopedia of Russian Internet Folklore (http://lurkmore.ru/%D0%9A%D0%BE%D1%80%D ... 0%BB%D1%8C). It was also a topic of a number of general essays in various periodicals, both computer and general ones. "Korchevatel'" (Russian for Rooter) is now used in internet blogs as a pejorative word for nonsense, gibberish, ravings, low-quality science etc. SCIgen is also rather popular.



At the end of the year, one weekly journal and one newspaper, both with reasonable reputation and circulation, recognized the story as "anti-scientific event of the year" (or is it "scientific anti-event of the year"?).


[ view entry ]   |  permalink  |  print article
Students at Sharif University get a paper accepted 
I just wanted to take a moment to sum up a little SCIgen-related story out of Iran that's come to my attention. PhD students at Sharif University, apparently fed up with their school's policies regarding how published papers are considered for their thesis requirements, used SCIgen to submit a paper to the Elsevier journal, Applied Mathematics and Computation:

Cooperative, Compact Algorithms for Randomized Algorithms
by Rohollah Mosallahnezhad, Iran Institute of Technology

I'm told that the author's name roughly translates to "from armed breed (race)" in Farsi (apparently as a jab against the European and US editors about the alleged nuclear threat of Iran), and that the institution is fictional as well. Anyhow, as you might guess, this paper was accepted by the journal as an "article in proof," and they even provided an edited and formatted article proof, along with an awesome set of corrections to be resolved. For several weeks it was listed as being in an editing phase.

However, the Iranian students felt that by getting the paper accepted, they had made their point, and announced to their colleagues that the paper was accepted, as proof to their university that the AMC journal did not really have any quality control, and that publications in that journal shouldn't be considered as seriously when judging the performance of PhD students. Presumably, one of these colleagues alerted the journal to their error, and now the journal has retracted the acceptance of the paper, complete with a note of apology from the editor.

--Jeremy

[ view entry ]   |  permalink  |  print article
Another SCIgen publication 
I just received word of a new SCIgen-based publication, this time in a Turkish media design conference. It was submitted by Professor Genco Gülan of Yeditepe University in Turkey. The title of the paper is "I/O Automata No Longer Considered Harmful" describing a system called Thumper, and it appears in the proceedings of the 3rd International Symposium of Interactive Media Design, pages 103-107, published in 2005 by the Yeditepe University Publication House. The conference website is here; however the online proceedings don't seem to include the paper. Professor Gülan was kind enough to mail me a print proceedings of the conference, however, and the paper is indeed in there.

Gülan made a presentation on "Problems of Interactivity in Education" at the conference last winter, and then he generated this paper while they were preparing the print proceedings and submitted it "as an example: both to show the problems of academy and the possibilities of electronic interactivity." The author also added his own digital art works at the end of the paper, to give reference to visual arts. An amazing thing about this paper is that Professor Gülan added an endnote to the paper explaining that it was randomly generated by SCIgen, and it was still accepted! According to the good professor, this is a "OK" conference in his area in terms of quality, but apparently they don't read their submissions.

Of course, even if they had it read it, I'm sure it would have been accepted anyway. I mean, how could they refuse a paper where the authors "added 8kB/s of Ethernet access to UC Berkeley's mobile telephones to better understand methodologies"? That's just good science.

- Jeremy

[ view entry ]   |  permalink  |  print article
International Symposium on Knowledge Communication and Conferences: KCC 2006 
Our good friend Nagib Callaos is holding more conferences again this year, all co-located together with WMSCI 2006 in Orlando, this July. The one that stands out in the crowd, though, is KCC 2006, which is "a symposium on the general theme of Knowledge Communications and its specific applications to the area of conferences’ organization, purposes, objectives/means, quality standards, etc." See: http://www.iiisci.org/KCC2006. [Warning: You may be sucked into a massive irony vortex shortly after opening this URL.]

I encourage everyone to submit a paper explaining their thoughts on how knowledge can be more effectively communicated at conferences such as these.

Jeremy

P.S.: Professor Callaos appears to have learned something about good figure design from our SCIgen diagrams. Check out the informative block diagram from the top of the KCC site:




[ view entry ]   |  permalink  |  print article
WMSCI 2005 
Giving randomly-generated talks was the most important reason we travelled down to Orlando in the middle of July; however, checking out the world-famous SCI multi-conference was a close second. The conference began with a "plenary breakfast" early on Monday, July 11th. This included breakfast for the attendees, as well as a keynote address from an esteemed researcher. Though we couldn't get inside the room without registering for WMSCI, we listened from the hallway.

We were expecting Nagib Callaos to give an introduction of some sort, but instead it was program commitee chair William Lesso, a long-time Callaos cohort who is apparently a professor emeritus at UT Austin. He welcomed everyone to the conference, and spouted on about how exciting it was going to be. He was thrilled about the reception banquet that evening, a new addition to the WMSCI schedule, which was going to be "the focal point of the conference." He warned the audience not to eat dinner, as there would be plenty to "graze on" at this reception.

There was no mention of SCIgen.

Finally, Lesso introduced the morning's keynote speaker, Gheorghe Benga. His webpage argues that he was cheated out of the Nobel Prize for his work on water proteins, and he has an astounding 450 publications under his belt (check out his CV). Of course, it's not too difficult to pick up 450 publications when you organize invited sessions at WMSCI that include 3 of your papers.

Gheorghe Benga's speech was on the topic of cybernetics and autism. Yes, you read that correctly. In a conference so general that it publishes over a thousand papers a year, the keynote address was on a topic so obscure that it might as well have been fictional. One has to wonder how many people at that breakfast could possibly be following along.

After a brief break, the technical sessions got underway. The day was broken up into 3 2.5-hour sessions, each with about 15 simultaneous tracks. In each session, between 4-8 papers. Some of the best session titles: "Development Methods and Methodologies," "Simulation and Applications of Modeling," "Key Words Finding Technique in Chinese Text Files" and "Computing Techniques."

There were a surprising number of people at the conference, given the amount of bad publicity it received, at least a few hundred in total. We saw between 4 and 20 people in each room. Most rooms did not have digital projectors so speakers had to make do with overheads; the digital projectors that were present were most likely provided by the speakers themselves. We know from experience that the AV package including the projector is several hundred dollars more that the one without, so this is probably quite an effective profit-saving technique employed by WMSCI at the expense of their attendees,

After the morning session, we talked to one man who had attended one of the sessons, and asked him how it was. He said it was all right, but that he was taking the rest of the day off to go to Universal Studios. A perfectly rational decision.

We made some badges for ourselves (not forged copies of WMSCI badges, just ones that looked somewhat official so we could walk around without too much suspicion), and attended the networking session in the early afternoon. It's probably not too productive to harp on the quality of the talks, so let's keep this brief. The talks we saw were fairly vacuous, without any major decipherable insights. They may or may not have represented good work, it's honestly hard to say. We asked some fair questions, and got back entirely orthogonal answers. At least one speaker did not show up (though we were late to the session).

And that was about all we could stand of WMSCI 2005. We flew back to Boston, satisfied and a bit melancholy. There was some disappointment in there, too: we didn't get thrown in jail, there was no angry confrontation with Nagib Callaos (we didn't even get a picture!), and we hadn't been able to give a fake talk at a real WMSCI session. But even so, we had succeeded in grabbing a lot of entertaining video for the folks back home, pissing off the WMSCI brass even more than we already had, and living to tell about it.

We're already looking forward to WMSCI 2006. Orlando, here we come!

--Jeremy

[ view entry ]   |  permalink  |  print article

| 1 | 2 | 3 | Next> Last>>