I’m starting a new R User Group in Doha, Qatar. Our first meetup is on 26th May, at the HBKU Student Center in Education City. I’ll be talking about run-time testing with my assertive package, and there will be two other speakers who I need to find pretty sharpish. (If you want to talk, get in touch!)
Both new and more seasoned useRs are welcome. RSVP on the meetup site:
(Registration is free.)
R has been translated into 20 languages but currently not many packages have translations. In a survey of CRAN done last December, of the 8274 packages on CRAN, only 50 had any installed translations. Of those, 28 had only a single translation. As the plot below shows, the number of translated packages is almost indistinguishable from zero.
The RL10N project by myself and the excellent Thomas Leeper has just been funded by the R Consortium in order to address this problem, and help R users who aren’t native English speakers. In short, we want to ASSIST R TO TAKE OVER THE WORLD (of data analysis). Cue evil laugh.
There are three strands to the project:
Firstly, there are tools in the
tools package for working with translations, but they are a bit fiddly to use. Thomas has a work in progress package called
msgtools that aims to make things easier. We’ll develop this package to be robust, well documented, and easy for novice package developers to use.
MTurkR package provides an interface between R and Amazon’s Mechanical Turk API, a marketplace for human intelligence tasks (HITs), including translation tasks. We’ll develop a package that wraps
MTurkR, with functionality for creating and managing translation HITs.
Thirdly, Christopher Lucas and Dustin Tingley’s
translateR package provides an interface to the Application Programming Interfaces (APIs) for the Google Translate and Microsoft Translator services for automated translation of text. We’ll create an R package that wraps
translateR, with functionality for integrating the automated translations into a package development workflow.
One of the bits of feedback that I got from the useR conference was that my
assertive package, for run-time testing of code, was too big. Since then I’ve been working to modularise it, and the results are now on CRAN.
Now, rather than one big package, there are fifteen
assertive.* packages for specific pieces of functionality. For example,
assertive.numbers contains functionality for checking numeric vectors,
assertive.datetimes contains functionality for checking dates and times, and
assertive.properties contains functionality for checking names, attributes, and lengths.
This finer grained approach means that if you want to develop a package with assertions, you can choose only the parts that you need, allowing for a much smaller footprint.
pathological package, which depends upon
assertive, gives you an example of how to do this.
assertive package itself now contains no functions of its own – it merely imports and re-exports the functions from the other 15 packages. This means that if you are working with
assertive interactively, you can still simply type
library(assertive) and have access to all the functionality.
Microsoft have announced new funding for R User Groups (Python and Hadoop too), so now seems as good a time as any for me to stop procrastinating and set up a Qatar R User Group.
If you live anywhere near Doha, and are interested in coming along to a (probably monthly) meetup about R, then fill in the survey to let me know when and where is best for you to meet, and drop me an email at richierocksATgmailDOTcom to say you’d like to come along.
I had an interesting email today saying that developers at the writer’s company wanted to use one of my packages, but weren’t allowed because it was under an unlimited license.
I release quite a few of my packages under an unlimited licence, at least for toy projects and immature packages. In those cases, letting users do what they want is more important to me than the fairness of, for example, the GPL.
assertive is a notable exception, because it’s taken a lot of work, and also because it contains some RStudio code.)
Anyway, the lady who wrote to me requested that I release my package under a dual license to enable her staff to use it.
My alternate solution is more elegant: since the license is unlimited, you can simply download the package source, edit the DESCRIPTION file to change the license to whatever you want, and use it as you see fit.
I gave a tutorial at useR on testing R code, which turned out to be a great way of getting feedback on my code! Based on the suggestions by attendees, I’ve made a big update to the package, which is now on CRAN. Full details of the new features can be access in the
?changes help page within the package.
Also, the slides, exercises and answers from the tutorial are now available online.
“Assertion” is computer-science jargon for a run-time check on your code. In R , this typically means function argument checks (“did they pass a numeric vector rather than a character vector into your function?”), and data quality checks (“does the date-of-birth column contain values in the past?”).
The four packages
R currently has four packages for assertions:
assertive, which is mine;
assertthat by Hadley Wickham,
assertr By Tony Fischetti, and
ensurer by Stefan Bache.
Having four packages feels like too many; we’re duplicating effort, and it makes package choice too hard for users. I didn’t know about the existence of
ensurer until a couple of days ago, but the useR conference has helped bring these rivals to my attention. I’ve chatted with the authors of the other three packages to see if we can streamline things a little.
Hadley said that
assertthat isn’t a high priority for him – dplyr, ggplot2 and tidyr (among many others) are more important – so he’s not going to develop it further. Since
assertthat is mostly a subset of
assertive anyway, this shouldn’t be a problem. I’ll take a look how easy it is to provide an
assertthat API, so existing users can have a direct replacement.
Tony said that the focus of
assertr is predominantly data checking. It only works with data frames, and has a more limited remit than
assertive. He plans to change the backend to be built on top of
assertive. That is,
assertr will be an
assertive extension that make it easy to apply assertions to multiple columns in data frames.
Stefan has stated that he prefers to keep
ensurer separate, since it has a different philosophical stance to
assertive, and I agree.
ensurer is optimised for being lightweight and elegant;
assertive is optimised for clarity of user code and clarity of error messages (at a cost of some bulk).
So overall, we’re down from four distinct assertion packages to two groups (
assertive). This feels sensible. It’s the optimum number for minimizing duplication while still having the some competition to spur development onwards.
The assertive development plan
ensurer has one feature in particular that I definitely want to include in
assertive: you can create type-safe functions.
The question of bulk has also been playing on my mind for a while. It isn’t huge by any means – the tar.gz file for the package is 836kB – but the number of functions can make it a little difficult for new users to find their way around. A couple of years ago when I was working with a lot of customer data, I included functions for checking things like the validity of UK postcodes. These are things that I’m unlikely to use at all in my current job, so it seems superfluous to have them. That means that I’d like to make
assertive more modular. The core things should be available in an
assertive.base package, with specialist assertions in additional packages.
I also want to make it easier for other package developers to include their own assertions in their packages. This will require a bit of rethinking about how the existing assertion engine works, and what internal bits I need to expose.
One bit of feedback I got from the attendees at my tutorial this week was that for simulation usage (where you call the same function millions of times), assertions can slow down the code too much. So a way to turn off the assertions (but keep them there for debugging purposes) would be useful.
The top feature request however, was for the use of pipe compatibility. Stefan’s
magrittr package has rocketed in popularity (I’m a huge fan), so this definitely needs implementing. It should be a small fix, so I should have it included soon.
There are some other small fixes like better NA handling and a better error message for
is_in_range that I plan to make soon.
The final (rather non-trivial) feature I want to add to assertive is support for error messages in multiple languages. The infrastructure is in place for translations (it currently support both the languages that I know; British English and American English), I just need some people who can speak other languages to do the translations. If you are interested in translating; drop me an email or let me know in the comments.