FLOSSmole - Collaborative collection and analysis of free/libre/open source project data

FLOSSmole is a meta-repository, created in 2014, of Free Libre Open Source Software projects data, which allows getting, using and/or donating Open Source project data with the next aims:

• freely provide data about free, libre, and open source software (FLOSS) projects in multiple formats for anyone to download;
• integrate donated data & scripts from other research teams;
  
• provide a community for researchers to discuss public data about FLOSS development.

FLOSSmole itself collects also data from software development forges, although, as explained before, it also accepts data donations.

In order to get data available in FLOSSmole, three options exist:

1. Flat delimited files
   Available in link FLOSSdata. The files are named with the abbreviation of the forge or source (e.g. Sourceforge is "SF", GitHub is "GH", and so on and so forth).
2. SQL files (mysql creates and inserts, generated from mysqldump)
   Available in next link: FLOSSdata. These files all start with the word "datamart", then the forge abbreviation, then the date. 
3. Direct database access
   Request access by joining the FLOSSmole mailing list.

Regarding the usage of the data, some resources are available as well:

• Database schema documents to help on figure out data available.
  
• Data collection details. An additional source of documentation about each of the sources of data existing on FLOSSmole.
  
• Examples such as visualizations and queries demonstrate potential uses of the FLOSSmole data.
  
• Research publications related FLOSSmole.
  
• Mailing list is also a good help to start on data usage. 

Several data  covering the period 2004 to present are available for usage, and growing with data sets from over 1500 web-based collection operations, including data for millions of open source projects and their developers.

What are you waiting to start using them?

Bitergia - Software analytics for your peace of mind

In previous posts, MetricsGrimoire and FLOSSMetrics were presented to show how already existing Free/Open Source Software Projects already exist in order to obtain metrics and try to evaluate the Quality and State of the Art of software for a particular project, whether it is FLOSS or not.

However, these tools, that are focused on retrieval and storage of the projects on a Database in order to created structured data, do not focus on the analysis, presentation or exploitation of the data.
 For this reason, the company Bitergia, has created a set of tools that enable, on the one hand, an analysis framework, written in GNU-R, called VizGrimoireR, and, on the other hand, a Dashboard-like web presentation of the metrics analysed, through the VizGrimoireJS library. A demo can be observed on this link.

These tools, together with MetricsGrimoire, help on making the circle narrower in order to generate complete reports to help on diagnosing, analysing and improving software projects (Free/Open Source preferably, but not necessarily).

Examples of the kind of reports being generated are, for example:

Mediawiki: http://bitergia.com/public/previews/2012_11_mediawiki/
Webkit: http://bitergia.com/public/reports/webkit/2013_01/index.html
FusionForge: http://bitergia.com/public/previews/2012_11_fusionforge/
OpenStack (Havana Release): http://blog.bitergia.com/2013/10/17/the-openstack-havana-release/
LifeRay: http://blog.bitergia.com/2013/10/23/liferay-and-its-growth-during-the-last-years/

This company, which has still a long way to go, above all in the Private Companies environment, rather than Open Source Projects, is also a good example on how to provide services through Free/Open Source Software, justifying that business models around Open Source are possible to implement.

FLOSSmetrics

FLOSSmetrics (Free/Libre and Open Source Software Metrics) was a Project, funded by the European Commission, whose main objective was to retrieve and analyse a big database with information and metrics about Free/Open Source Software projects using already existing methodologies, and relying on already developed tools.

The project achieved the retrieval of more than 2800 FLOSS projects in Melquiades (lamentably down nowadays), by using existing Open Source tools such as CVSAnalY.

Among the most important broadcast activities, a talk in FOSSDEM 2009.

The main objectives of the project were:

- Identification and evaluation of sources of data.
- Creation of a comprehensive database from previous sources.
- Integration of already available tools to process data and create a complete platform for its availability.
- Maintenance of the empirical database by automation of extraction of data of thousands of Open Source projects with extraction tools.
- Develop of studies, (benchmarking, simulations and comparisons of FLOSS projects).
- Propagation and exploitation of the results.

To achieve the objectives, delivery of next Work Packages (WP) was performed:

• WP1 - Data Sources
• WP2 - Retrieval System
• WP3 - Database
• WP4 - Analyses
• WP5 - High Level Studies
• WP6 - Visualization
• WP8 - SMEs
• WP9 - Industrial Exp./Val.
• WP10 - Exploitation
• WP11 - Productivity

Among the different documents resultant of the project, FLOSS Guide for SMEs (small and medium enterprises) is a very interesting one, advent to present a set of guidelines and suggestions for the adoption of open source software within SMEs

On 2010, elaboration of a Final Report  concluded this very interesting and fruitful collaboration project.

Metrics Grimoire - A toolset to generate software metrics

As time goes by, it has been demonstrated that software metrics are important to categorize the status and health of a particular project, independently of the reason that it is or not an Open Source project.

Metrics Grimoire toolset allows to obtain certain amount of parameters from different kind of sources to generate metrics that enable analyzing a software project.

Among the different tools that the toolset provides, next ones are the most remarkable:

CVSAnalY: Do not be confused because of its name. CVSAnalY project analyzes different kind of repositories (CVS, as well as Subversion and GIT) and stores analyzed information on a SQL database. This information can be later analyzed in order to measure, for example:
- Most active committers.
- Number of commits in a period of time, or a a particular time or date.
- Territoriality of the project, by means of calculation of the different people modifying a particular file.
- And, in general, all the information that can be retrieved from that particular Software Configuration Manager.

Bicho: This tool allows to perform analysis on different Bug Trackers (Bicho means "Bug" in Spanish). The tool enables browsing different kind of Issue Trackers, and store some data from then in an SQL Database. Among the supported Issue Trackers, next ones are found:

- Bugzilla
- Launchpad
- Jira (unstable, partially supported)
- Allura (unstable, partially supported)
- Github (unstable, partially supported)
- SourceForge (abandoned)

The tool allows to obtain metrics such as:
- Number of issues of a particular state.
- Number of issues whose state changed on a particular period of time.
- Top Issue Openers/Resolvers/Closers
- People who have been assigned with a higher amount of Bugs
- And, in general, all the information that can be retrieved from that particular Issue Tracker.

MlStats: This tool allows to retrieve information from a mail list. It works basically with mailman. The tool allows to analyze information such as:
- Number of messages in a day for a particular mail list.
- Top askers and responders.
- % of messages responded.
- And, in general, all the information that can be retrieved from mailman mail list system.

To summarize, a very complete set of tools, which are Free Software (as in Freedom), which are costless, and wich enable to provide analysis of Open Source projects, but also Proprietary software on companies using previous described sources.

Motivating community for starting Open Source Projects

Open Source Projects are, indeed, not far from other type of associations or companies where motivation is a key.

Community building is important for already existing projects with strong communities behind.

However, starting Open Source projects are also challenged to show what the project offers, the kind of beneficiaries this project should have, technology , members of the project, and so on and so forth.

I would like to share a nice motivating video on how to "sell" to the community a starting project, in this case, a project hosted by the Linux Foundation, the OpenDayLight Open Source Project [1]:

The video, from my perspective, it is interesting as a good example of community attractive video because of next reasons:

1 - It is concise.
2 - It shows the motivation of the project clearly.
3 - It is "smart" and graphically attractive.

You work as computer engineer or as network engineer. You know Big Data, Cloud Computing and SDN will be the future technologies. Don't you feel a little bit motivated to participate in the community after watching the video?

References:
http://www.youtube.com/watch?v=ZWzMT5KLEtY

Developers Motivations - Why start contributing to Open Source Projects?

There is a huge amount of software developers who always have worked on privative software, basically, for companies that produce software .

However, as an additional task, it is interesting for them to understand why they should start contributing to Open Source projects.

On next video, Jessica McKellar, Open Source advocate, core developer of Python and member of the Python Software Foundation, and maintainer for Twisted and OpenHatch [1],  explains, among other stuff, some clear reasons about why someone should consider, at least, start contributing to an Open Source project [2]:

From Jessica's perspective, contribution to Open Source projects should be performed in order to:

1) Give back to the community. Above all, if you like a particular technology or project, you should consider start contributing to it as a way of restitution.

2) People. As far as she is concerned, smartest people she knows comes from Open Source project movement.

3) Education.  With a Master on Computer Engineering, Jessica recognizes not many lessons being taken on Software Engineering. Contributing to an Open Source project is a way of enforcing knowledge on Software Engineering, as well  as obtaining experience.

4) Work. Open Source contribution is a way of transparency and a "do-ocracy" way of showing someone's capabilities to people and companies hiring.

5) Enjoying. In the end, contributing to a community ends being an enjoyable thing.

The video also shows a complete bunch of topics on FLOSS contribution, as different ways to contribute. Apart from that, Jessica also explains on a very simple way the different technical aspects common to many projects, such as the Bug Issue tracker, the ways of performing changes in code and generating a patch, etc.

References:
[1] http://web.mit.edu/jesstess/www/
[2] http://www.youtube.com/watch?v=IXnNgLmd6BM

FLOSS Foundations

Linux Foundation, Free Software Foundation, Apache Foundation, Mozilla Foundation and Eclipse Foundation could be categorized as the "Top Foundations" in terms of popularity. Hosting projects as the Linux Kernel, GNU tools, the Apache Web Server, Mozilla Firefox or the Eclipse Editor.

However, there are more Foundations apart from the previous ones. In fact, there is a whole bunch of Foundations, associated to the most successful Open Source projects. A good place to check them is the Foundations directory of http://flossfoundations.org/, where other important foundations can be found. Among Foundations associated to Open Source Projects, it can be found [2]:

- The Blender Foundation
- Django Software Foundation
- FreeBSD Foundation
- Gentoo Foundation
- GNOME Foundation
- KDE e.V.
- NetBSD Foundation
- OpenBSD Foundation
- The Perl Foundation
- PostgreSQL.US
- Python Foundation
- X.org Foundation

Besides this, we find other very important associations, not associated to a particular project, but to accomplish Open Source and Open-knowledge advocating and promotion. Examples of them are [2]:

- Wikimedia Foundation
- OpenSource Initiative
- Free Software Foundation
- Tex Users Group
- Creative Commons
- OpenDoc Society
- Subversion Corporation

Another category of associations are the ones having to do with regional promotion of FLOSS.  Examples of them [2]:

- Associaçao SoftwareLivre.org (Brasil)
- El Centro de Software Libre (Chile)
- Fundación Via Libre (Argentina)
- Free Software and Open Source Foundation for Africa
- Software Libre Argentina
- Software Livre Brasil
- Software Libre Chile

Some other Foundations are dedicated to promotion of open standards across the Web and Internet. The most remarkable examples are  [2]:

- Internet Software Consortium
- XMPP Standards Foundation
- Xiph.org

Last, but not least, other Foundations have to do with heterogeneous activities, such as Open Health promotion, Digital Rights defence,  Public-Domain works promotion, Social Development, etc. Examples of this kind of Foundations are:

- Electronic Frontier Foundation
- Shuttleworth Foundation
- OpenHealth Tools
- OpenHardware Foundation

In the end, a lot of Foundations appearing on the list.
However, which are the most active? Which of them have greater supporters? Who of them have strong companies behind?
A good parameter for evaluation has to do with revenues. No doubt the most successful project, the Linux kernel, has behind the Foundation with the higher revenues. A good article from Brian Proffitt for ITWorld, showed the revenues of the most important Foundations in the USA [3]:

The Linux Foundation: 2010 Exercise:  Revenues: 9,618,043$
Mozilla Foundation: 2010 Exercise: Revenues: 1,934,659$
Free Software Foundation Inc.: 2009/2010 Exercise: Revenues: 889,400$
Python Software Foundation: 2010 Exercise: Revenues: 706,136$
Gnome Foundation Inc.: 2009/2010 Exercise: Revenues: 636,648$
Apache Software Foundation: 2010/2011 Exercise: Revenues: 539,410$

In the end, the Linux Foundation, surely because of the great companies behind it.

To conclude, just highlighting that, although the numbers of Computer, Technology and Internet industry are huge, no significant amount of money is handled by the most important Foundations (Linux Foundation as exception), taking into account the importance of the projects behind them, as well as the important labour they accomplish.

Reference:
[1] http://flossfoundations.org
[2]  http://flossfoundations.org/foundation-directory
[3] http://www.itworld.com/it-managementstrategy/260688/nonprofit-open-source-organizations-booming