Keeping git-annex's momentum going strong

Last year I did a Kickstarter and asked for funding to spend a few months building the git-annex assistant. It was beyond successful, and I ended up developing it full time for a full year.

The easy to use webapp, ports to Android and Mac OS X, and Windows, and other great features I've worked on in the last year have brought git-annex to more users than ever before. Together we kick-started something awesome!

But now it's important to keep that momentum going. While lots of new people are trying out git-annex, I need to be able to devote all my time to fixing problems they find, and making it work really well.

Make no mistake, I'll continue working on git-annex no matter what happens here. The only question is, will I be able to give it my full attention in this critical period?

... Well, that question has been answered. I'm working on git-annex full time for another year! And maybe a little beyond.

Joey Hess

Ongoing development

There are so many features to add, and ways to improve git-annex, that I can't begin to list them all here, but here are a few:

  • Integrate better with Android.
  • Get the assistant and webapp ported to Windows.
  • Refine the automated stress testing tools to find and fix more problems before users ever see them.
  • Automatic recovery. Cosmic ray flipped a bit in a file? USB drive corrupted itself? The assistant should notice these problems, and fix them.
  • Encourage more contributions from others. For example, improve the special remote plugin interface so it can do everything the native Haskell interface can do. Eight new cloud storage services were added this year as plugins, but we can do better!
    See also: the git-annex enhancement contest
  • Use deltas to reduce bandwidth needed to transfer modified versions of files.
  • Improved ssh password prompting when setting up a ssh remote in the webapp.

Focus on security & privacy

Git-annex users are storing their files securely encrypted in the cloud. They're avoiding getting locked into one proprietary service. They're able to sync and share their files without losing control of their data. Recent developments have highlighted the importance of software that helps you protect your privacy and avoid lock-in.

I was thrilled to see that git-annex is featured on prism-break.org as an alternative to DropBox, Google Drive, and iCloud. Proud to see it in the company with so much other good software that respects these principles. But also worried, since any software using encryption always has room for improvement.

So, this campaign has an additional focus on improving git-annex's security. Some of the areas I want to work on:

  • Add easy support for encrypted git repositories using git-remote-gcrypt, so you can safely push to a repository on a server you don't control.
  • Add support for setting up and using GPG keys in the webapp.
  • Add protection to the XMPP protocol to guard against man in the middle attacks if the XMPP server is compromised. Ie, Google should not be able to learn about your git-annex repository even if you're using their servers.
  • To avoid leaking even the size of your encrypted files to cloud storage providers, add a mode that stores fixed size chunks.

Blog

the gift that kept on giving
Posted
the year you made happen
Posted

FAQ

How do I send my donation?

You can support me with a donation, made using Paypal, a direct transfer from your Bitcoin wallet, or with Bitcoin purchased via CoinBase. Just click on a button for the level you want to donate.

You can also send me donations of any size directly. Send PayPal donations to joey@kitenet.net. To avoid all payment processing overhead, you can send Bitcoin directly to me at 1joeygoqqHtkxkQ3MRnhDZsBXHqUNbEb9

Is this the best way to fund ongoing git-annex development?

This is a one-time thing. I do not plan to try to live off of the generosity of my backers indefinitely. But it's sure been great to spend a year building exactly what I feel needs to be built, with the ongoing feedback of my backers!

I do have some other Kickstarter-worthy ideas for projects that might involve git-annex in other ways, and might lead to a sustainable funding in the long term. I have not decided yet if I'll try to take such a project on.

Also, I've heard from a ton of people encouraging me to run another fundraiser, both happy Kickstarter backers, and people who missed their chance to contribute to the first one. Who am I to say no to that? ;)

When does the fundraising campaign end?

The initial phase ends on August 15th.

Since this is a somewhat open-ended campaign, I will extend it if the initial goal is met and it seems appropriate to do so.

When will I receive my reward?

The sponsored commits and the day's sponsor will happen in the order people signed up.

I'll work out a time to do the personal git-annex assistant gig, depending on when we both have time.

Stickers will be mailed out within 2 months.

Why should I give to git-annex rather than to some other deserving free software project?

One of the great things to come out of my original git-annex Kickstarter, that I did not anticipate, was that it helped inspire some other similar attempts. MediaGoblin had a successful croudfunding campaign. There is an ongoing campaign by the Software Freedom Conservancy to fund development of Non-Profit Accounting Software.

The Free Software community is still figuring out how to use crowdfunding effectively. Does it make sense to continue supporting a project that's already had one successful round of crowdfunding? I think it's best to leave the decision up to you!

Can I sponsor multiple commits, or get multiples of other rewards?

Sure, just donate $50 x 2 = $100 to sponsor two commits, or combine other sponsorship levels. I'll get in touch if I can't figure out what you wanted based on the amount you donate. ;)

Is this donation tax deductible?

No, it is not. However, I have structured this campaign to avoid other overheads that use up significant percentages of the funds raised by a typical Kickstarter project. That's one reason why I'm encouraging donations using Bitcoin.

What is the software license?

git-annex is GPL-3+, and its webapp is AGPL-3+.
This web page is CC-BY-SA 3.0.

What people are saying about git-annex

Git-annex now auto-syncing photos from my android phone to a Tor hidden SSH service I control (via @guardianproject's Orbot) #prismbreak
  — Richard

Annex is working great for me since 8/2012 or something... Using it a lot and the Assistant is awesome! really great software!
  — Subito

Git annex is great to keep all my repository synchronized while not requesting that all my pictures are present on my laptop. Thank you!
  — Nicolas

I discovered git-annex few months ago and it really helps me to keep my files well organized. I missed the first crowd funding campaign but would be happy to contribute to the next one :)
  — Julien

I've finally gone from either keeping hard drives from my past computers or keeping a full copy of the data, to having a proper archiving and backup system for all files I care about. I've got over 12,000 files stored on my main annex, and it's perfectly responsive.
  — Jason

Git-annex makes it possible for me to have lots of data. Before I started using it I had a music library and a few massive tar files, none of it properly backed up, and occasionally there would be random corruption (I still have broken mp3s which jump and stutter in places) because I would carelessly cp them from HDD to HDD. So I'd avoid actually letting data accumulate; this was inconvenient in itself. Now git-annex does everything for me.
  — anonymous

I'm using git-annex to consolidate all the stuff that accumulated for years on various drives I have. Music I downloaded and bought (don't wanna loose those), photos from 2003 and on, archives of school and work stuff etc. Using it for 6-7 months now, I came to trust git-annex

Thanks Joey for all the great work, I really hope you can continue improving it!
  — dzsino

I'm a graduate student, so I write a lot. I started using git to manage my latex sources a while ago, but I didn't like gumming up my repository with a bunch of figures (large binaries). I was constantly forgetting to scp the latest versions of my figures from here to there (as I do my work on a number of different computers.) Git annex gives me a good way to version the figures and manage them with the same tool I'm using for the text.
  — Matthew

I'm using git-annex to make videos, pictures and music available on all my devices (windows, android, linux). It has saved me on trips with my 2yr daughter when she's had meltdowns; I have her favorite movie ready. I quickly exceeded the free dropbox quota and git-annex has replaced the need and gives me greater confidence and control over my data. Thank you!
  — Joe

I have moved all my photographs to an annex running on a Ubuntu file server while doing photo editing, presenting etc. on my MacBook. The annex is great in selectively moving files between these machines for editing and presentation purposes. I started using it around Christmas season last year and added a glacier "backup" recently. My statistics: 6 repositories in total, more than 300GB (dedup'd from 380GB)
  — Hendrik

I sync my working machines, my home fileserver, my VPS (for file access on the go), Android devices, and even a couple of dumb MP3 players from git-annex. Best of all, I have one place to go for all of my files, and if they're not on the device at the moment I need them, they're a download away.
  — Evan

I've got an annex with all my DVD ISOs that's nearly 3TB alone, uses the SHA512 hash and gets split over three computers, 6 external SATA drives and 3 USB drives.

If you do another fundraiser you've got my support!
  — glittah

I have 6+ repos, 235 GB of data; nevertheless git annex is generally pretty cooperative. I almost don't have to think about the stuff annex handles. It's really fun to install a fresh OS, clone the git annex repo, and have my entire annex directory tree available immediately.
  — Andy

Our 4T of data gets deduplicated to about 270G, only a couple of megs get transferred every night, and thanks to git-annex everything just works, like it should!
  — Tim

Thanks very much for the dev blog! I was looking forward every morning to see what's coming next, what problems you are facing and how you're planning to solve them. It is simply amazing what you've achieved in the last year and your posts gave me a lot of motivation
  — David

I know that my expectations in this project are a little bit high and I see how much work it was to get where we are now. Once again thank you for the excellent work done so far. I don't fear to synchronize important data with git-annex.
  — Florian

git-annex とか実際どうなんだろうなあ。
tweeted