The TPF Inline Grant Info Site

Up to date information on The Perl Foundation's 2014 grant for Inline.pm.

Inline Grant Wrap Up (and Christmas Delivery)

Ho ho ho! Merry Christmas!

Regardless of your holiday faith inclinations, in the Land of Perl it's all about Christmas. That's the day that projects and promises get wrapped up and delivered. Today Ingy and David (döt Claus) deliver you a fresh new Inline::Module! It's the culmination of 2 busy months of coding that wraps up our TPF Grant work.

This is the final report for the TPF Inline Grant. It tells of the journey; the good, the bad and the awesome; and ends up with the shiny new gift under your CPAN tree. You also get your fair share of stocking stuffers. Read on!

Inline::Module

When David and I started the grant work, we knew what we wanted to accomplish, but we really didn't know how we were going to do it. We didn't even know what the new modules would be called. Here's what we did know:

  • We wanted Inline modules to be on par with XS ones
  • We wanted no extra dependencies for installation
  • We wanted to support all common Perl module frameworks
  • We wanted Inline usage to not change much
  • We wanted the new framework abstractions to feel right
  • We wanted to support C++ as well as C

Two weeks before we began I asked a bunch of smart people at an Amsterdam.pm meeting, how they might go about it. I don't recall all the ideas that were talked about but I do recall that leont++ was there. Leon ended up being most helpful on several occasions for this project.

Once we started, we quickly settled on making a new helper module called Inline::Module. All the new logic was added there. We wrote plugins for 3 of the frameworks, but all the logic for them is in Inline::Module.

Inline::Module is less than 500 lines. The plugins are all under 100 lines. The Inline module itself required one small change. Inline::C was not changed at all. Inline::CPP need a few changes to bring it more up to date with Inline::C.

The plan was pretty simple. We needed to modify the build and distdir operations of the module frameworks. We also needed to make sure that a prove -l t/ command would trigger a build. That's pretty easy, because that's what Inline already does. We just needed to make sure that everything happened at the right times (the same times that an XS build or distdir would do them).

Mine's a Stubby, Mate

The biggest problem was how to make sure that an installed module based on Inline never needed Inline.pm and friends. If you say:

use Inline C => <code>;

then you absolutely will load a module called Inline. That's the way all Inline usage happens (in a use statement), so what could we do?

What if we made up a new convention to replace Inline with a module that sometimes called Inline (during development, testing, and user side compilation) and then only ever called DynaLoader after installation (no matter what)? Sounds worth a try, right?

use Your::Module::Inline C => <code>;

That's the new convention. Make a new module by adding ::Inline to the end of whatever module is using it. This module will proxy its arguments on to the real Inline (or just call DynaLoader after installed). We call this new module a stub. I'm not sure if that's the best name, but that's the name we chose.

The stub module is just a few lines long. Actually there are 2 distinct stub forms: the Inline one and the DynaLoader one, and they are both a few boring lines long. So boring that Inline::Module generates them for you.

The explicit way to do it is with this command:

$ perl -MInline::Module=makestub,Your::Module::Inline
Created stub module 'lib/Your/Module/Inline.pm' (Inline::Module 0.30)

Or you can add makestub => 1 to your Makefile.PL and it will keep your stub(s) up to date every time you run it.

We also created a technique called autostubbing that makes an in-memory-only module, so you don't have the generated code on disk. It requires that you set the PERL5OPT variable a certain way. Looking back, this technique might not be our best practice and we may remove support for it in the future.

Using Inline::Module

We really wanted to make sure that this new stuff worked in whatever module framework people wanted to use. Dist::Zilla is popular these days, but there's also Module::Install, Module::Build and plain old Makefile.PL (ExtUtils::MakeMaker). I (being Ingy) also have a new one that I use exclusively called Zilla::Dist. Inline::Module supports them all.

With EU:MM you just use Inline::Module in your Makefile.PL. D:Z, M:B, and M:I all have plugins, since that's how they work. Z:D support is built-in. Z:D just generates a D:Z dist.ini file from metadata (a Meta file), so effectively it uses the D:Z plugin as well.

All the plugins are simple and do the same things but for their respective frameworks. To make sure that there was no duplicate code, the plugins call methods within Inline::Module to do their work.

Inline::Module just needs 3 things to do its magic:

  1. The module(s) that has the inline C/C++ code
  2. The name(s) of the stub(s)
  3. The name of the Inline Language Support Module (ILSM)
    • Inline::C
    • Inline::CPP

The stub names default to appending ::Inline to the module names. The ILSM defaults to Inline::C.

In a Makefile.PL you add something like this to the WriteMakefile arguments in your Makefile.PL:

postamble => {
  inline => {
    module => 'Acme::Math::XS',
    stub => 'Acme::Math::XS::Inline',
    ilsm => 'Inline::C',
  },
},

With Dist::Zilla, you add the same info to your dist.ini file:

[InlineModule]
module = Acme::Math::XS
stub = Acme::Math::XS::Inline
ilsm = Inline::C

and so on. Each framework takes the same info.

Re: Module::Build: There was some push-back from the community to provide support for Module::Build because some people think it is dead, or want to see it die ASAP. I had no opinion one way or the other, but when you have the primary maintainer (leont) hanging out and offering help, why not give it a try? In the end, I found interacting with Module::Build to be a very pleasing experience. Take a look at the code. Not bad, eh?

Public Displays of Affection

I love the Perl community because of its fantastic group energy. Nothing is impossible. Everything's a challenge. I've always tried to take the highest, boldest, freshest, and most impossible roads to programming, and no other community is more accommodating than Perl's.

I also like to push the boundaries of group and public programming. When I got this grant, I wasn't going to waste the opportunity to push forward on multiple fronts. Even before the project was granted, I decided that /I/ should be we. This is the first TPF project to be granted to a pair of programmers. Hopefully that continues.

But why stop at 2? We wanted to involve as much of the community as possible, and use the most transparent methods we could think of. Not everything we did was a total success, and I'm sure we could have taken it much further but here's some of the things we tried:

  • Communicate entirely in IRC (irc.perl.org #inline)
  • Use PairUp™, remote shared server using tmux and weechat
  • Televise (live stream) the PairUp™ sessions using termcasting
  • Write weekly blog posts on our progress
  • Tweet and post to http://blogs.perl.org
  • Notify the Perlweekly newsletter
  • Continually release to GitHub and use their issues tracker
  • Invite framework experts (ether, leont, sivoais) into PairUps as needed

The IRC channel was a big success. There's about 20 people in there on average, and they are quite the knowledgeable crew.

PairUp is a technique I've used for over a year now to pull unsuspecting IRC/GitHub users into a live coding session. I love everyone. You're next!

We gave up on the termcasting due to numerous technical glitches. I'd like to see it fixed, because it lets anyone watch the live pair programming terminal (but not interact with it). They can then see what we are doing and offer help on IRC. Part of the PairUp terminal is an IRC pane that is always tuned to #inline.

David and I are Inline experts, but we aren't module framework experts. (I've written a few, but I'm no expert). When it came to Dist::Zilla and Module::Build we needed help. We asked for help. We got help. In real time! We invited the framework experts to #inline and asked them what to do. When things went south, we got them into the PairUp session and they set things right within minutes.

A shout-out goes to (at least):

doy
Termcast fixes
leont
EUMM & M:B help
sivoais
Pairing and using I:M
ether
dzil help
priodev
Always up for a PairUp!
rafl
Suggesting XS modules to look at
bingos
Fixed BSD/CPANPLUS smoker bug
mohawk
Various ideas
bulk88
Windows and C expertise
Mithaldu
More win32 help
mst
For being mst

Stocking Stuffers

Building a new Perl framework is only theory. Releasing CPAN modules that use it makes things real. Many modules have been released using Inline::Module.

We started off making a simple XS module called Acme::Math::XS. It has add() and subtract() methods/exports. As simple as it gets.

Using the Alt methodology, we made 7 Alternate versions:

Alt::Acme::Math::XS::EUMM

Version using Inline::Module with a Makefile.PL and ExtUtils::MakeMaker.

Alt::Acme::Math::XS::DistZilla

Using Inline::Module with Dist::Zilla and Dist::Zilla::Plugin::InlineModule.

Alt::Acme::Math::XS::ModuleBuild

With Module::Build and Module::Build::InlineModule.

Alt::Acme::Math::XS::ModuleInstall

With Module::Install and Module::Install::InlineModule.

Alt::Acme::Math::XS::ZillaDist

With Zilla::Dist.

Alt::Acme::Math::XS::CPP

Using Inline::Module with Inline::CPP and EUMM.

Alt::Acme::Math::XS::External

Using Inline::Module with an external C file.

All of these code bases are under one repository, using multiple branches: https://github.com/ingydotnet/acme-math-xs-pm

We also converted David's Inline::CPP example module Math::Prime::FastSieve to Alt::Math::Prime::FastSieve::Inline and an XS module suggested by Florian Ragwitz, Devel::GlobalDestruction::XS, to Alt::Devel::GlobalDestruction::XS::Inline.

Unexpectedly, other people started using Inline::Module, along the way. I guess that's what happens when you make new stuff! sivoais++ wrote something very ambitious called Statistics::NiceR that uses Inline::C to bind R to Perl. With very few bumps, he was able to use Inline::Module, even before it was done! He wrote a nice post about it here.

He (sivoais) also noted this project that is using Inline::Module: https://github.com/tadegenban/Text-Levenshtein-Inline-pm

Testing

In the past 2.5 years I've done a lot of serious Bash programming. I've written a full featured Git command that lets you do all your GitHub interaction at the command line, called git-hub. This required me (among other things) to write a JSON parser in Bash, and to port Test::More to Bash.

Inline::Module is a Perl module, and tests must be in Perl. Or must they? External tests (the ones that go in xt/) could conceivably be in any language. In fact, prove caters to this. It honors the hashbang line of the .t files.

I could have 20 .t test files in 20 different languages and (as long as they had the right hashbang and output proper TAP protocol) they would all work fine with:

prove -lv t/

For Inline::Module, most of what really needed testing was the human interaction at the commandline and the file system expectations. From experience I knew that expressing this using shell (Bash) is much easier and clearer than trying to do it in Perl.

Take a look at these test files:

I wanted to make sure that the all known Inline::Module modules got put through a common set of tests. The first file states all the modules, and their specifics. The second file has the Test::More testing in it.

I can't imagine doing all that more simply in Perl. Not even close. Pull requests to the contrary are welcome. :)

Swimming the Documentation

All the modules come with documentation but something big like this really needs a tutorial, so you get one: Inline::Module::Tutorial.

We wrote all the doc in a new markup language called Swim. Since May 2014, I've converted all my Perl Pod doc to Swim. Why?

Pod has a rich model for creating complex, informative, beautiful docs. Much richer than Markdown. So much so, that I really can't use Markdown. But to make nice looking HTML from Pod, takes a ton of work. Pod syntax sucks. Markdown has decent syntax that people like. Swim has syntax like Markdown (but even better) and produces everything that Pod can. So I write in Swim and publish in Pod. Since GitHub supports Pod (very well in fact) I can even publish Swim (as Pod) to GitHub!

Not only did we write all the doc and the tutorial in Swim, we wrote these blog posts in Swim too, using a new homegrown blog system we cobbled together. It's all here.

What's Next?

This project was a lot of work (100s of hours) but a lot of fun too. I think we've set all the balls in motion that we intended, but is the project "done"?

No way! Software projects are never done (done == dead), and this one is no exception. We gave birth to a new life form, but now it needs to grow up. Luckily it has you! The Perl community is the best family to raise a software child.

There are lots of things I can think of to make Inline::Module better. You'll see these in the forms of GitHub issues and CPAN releases.

Thank you TPF and Perl Community for giving us this opportunity. We hope it is as good for you as it was for us.

Merry Christmas!

Ingy and David


Inline Grant Weekly Report #9

It's just days before Christmas.

What happens on Christmas? It's the when Perl software is READY!

What does one do in the days before Christmas? Wrap things up, of course.

Want a present? Get Inline!

David and Ingy are trying to be good Santas and get our gifts for all the Perl kids wrapped up in time for the special day. This week the final pieces came together:

  • Module::Build Support
  • Inline::CPP compiler detection
  • Lots of testing, debugging and fixes

You'd better not pout. I'm telling you why... Inline::Module's coming to Perl!

Be nice!


Inline Grant Weekly Report #8

This was another big week for Inline::Module and friends. This is the week that modules using Inline::Module start getting out to CPAN. This even includes the first one written by someone, not working directly on the project! Read on.

So Many Modules!

This report will be organized by the modules created and updated this week.

R.pm

The hacker known as sivoais on IRC #inline, started a project to embed the "R" language in Perl. It uses PDL and Inline::C. This week he got the whole thing to work using Inline::Module and Dist::Zilla::Plugin::InlineModule. It's not yet shipped to CPAN, but the repo is here.

I suspect it will be on CPAN soon. The dzil build command produces a working dist that has Inline::Module and Inline::C support to do all the XS work easily. That's what this grant is all about. Thanks sivoais! (aka ZMUGHAL on CPAN)

Alt::Math::Prime::FastSieve::Inline

David wrote Math::Prime::FastSieve a few years ago, to show off using Inline::CPP for a CPAN module. He was able to do it, but it had an Inline requirement. (Ditching that requirement is a primary goal of this project).

This week we released it as Alt::Math::Prime::FastSieve::Inline using the latest Inline::Module. This required some refactoring of Inline::CPP. See below.

Inline::CPP

We needed to bring Inline::CPP into closer parity with Inline::C (in the parser department). Basically we moved Inline::CPP::Grammar to Inline::CPP::Parser::RecDescent. This also makes Inline::CPP work with Inline's using directive.

This let us do the right thing from Inline::Module and after that C++ Just Worked™. Note that even though Inline::CPP uses both Inline and Inline::C, no changes were required to those two modules. (No change, is good change).

Dist::Zilla::Plugin::InlineModule

In order for sivoais to use Inline::Module, we needed to update the Dist::Zilla plugin to work with the latest code. Ironically, sivoais himself did most of the work. Ingy pulled him into a realtime PairUp™ session and soon it was done. Dist::Zilla superstar ether++ helped us find a couple of the deep magic requirements that were needed to make it all work.

The Dist::Zilla::Plugin::InlineModule module ends up being extremely simple though. It just glues Inline::Module and Dist::Zilla together with no real special casing for either. The same code that does the right things to an EUMM (or other) dist, does it for Dist::Zilla. This should lead to maintenance happiness down the road.

Alt::Acme::Math::XS::*

Our test module Acme::Math::XS got released in it's various forms:

Next Steps

Our estimate for the grant work was 2 months. Next week will complete 2 full months since we started. We are very close to done, and will try to wrap things up if possible. The things that come to mind are:

  • Module::Build support module for Inline::Module
  • More tests
  • More Alt modules

Stay tuned!


Inline Grant Weekly Report #7

This week made great strides for Inline::Module. Last week, we talked about auto-stubbing. That's the automatic generation of little "stub" modules that proxy invocations of Inline, so we can write simple code like this:

package My::Module;
use My::Module::Inline C => '... C code here ...';

and have it do all the right things at the right times for My::Module. This obviously means that My::Module::Inline needs to exist somewhere, even though it is just a couple of lines of simple code.

This week auto-stubbing is a reality and it works well! In fact, it turns out that the "stub" module never even needs to exist on disk. Read on!

Latest Developments

We got rid of the bin script: perl-inline-module that was used to generate stubs, like this:

perl-inline-module generate Acme::Math::XS::Inline

We replaced it with a simple one-liner:

perl -MInline::Module=makestub,Acme::Math::XS::Inline

That will generate the file: lib/Acme/Math/XS/Inline.pm. You can generate it under the blib directory like this:

perl -MInline::Module=makestub,Acme::Math::XS::Inline,blib

Auto-Stubbing

I'd call that Explicit Stubbing. DON'T DO THAT!

Well you can if you want but the cool new way to use Inline::Module is with Auto Stubbing. Here's how you do it:

export PERL5OPT=-MInline::Module=autostub,Acme::Math::XS::Inline

Now whenever Acme::Math::XS::Inline is needed in development, it is provided/loaded as a Perl in-memory file object!

You can also auto-stub to disk if you want:

export PERL5OPT=-MInline::Module=autostub,Acme::Math::XS::Inline,blib

but why would you want to? With the in-memory stubs, you don't need to worry about an extra file laying around.

blib/Inline/

This is a small change but now all the Inline build time stuff happens under the blib/Inline/ directory. We had it building directly under blib/ but since that is a well defined concept, it made things confusing.

In general with this project, we are trying to extend Perl coding best-practices in ways that make XS module authoring as simple as possible, while not diverging very far from normal authoring styles.

Testing

We started writing tests that can verify all the processes we are imagining. Most of these tests so far are of the xt form. Since almost anything goes in xt I decided to write these tests in Bash instead of Perl. Since these tests are generally of the form: "run this command, in this environment and see if these files exist, etc", Bash tests make sense.

Some time ago I ported Test::More to Bash. You can run them with prove justlike Perl tests. The prove command just looks at the hashbang line of the .t files and sees that it is Bash, and runs Bash instead. Here's an example test file. As you can see it is very simple and easy to understand. If you squint your eyes, it almost looks like Perl!

Next Steps

We should be wrapping this Grant project up soon. We still need to:

  • Add Module::Build support
  • Update all the pluins
  • Update the docs
  • Migrate a few real XS modules to Inline::Module

Stay tuned. Inline Modules are becoming a reality!


Inline Grant Weekly Report #6

Happy belated Thanksgiving!!!

Last week was a bit slow on the Inline front. Not much code got done but one important idea came along that (potentially) simplifies the whole Inline::Module dance. We call it auto-stubbing.

Auto-Stubbing

When a module has Inline C (or C++) code in it, it needs a stub module to invoke Inline and/or dynaload a shared library. This stub is generated code. We had a command for the module auhor to generate it:

perl-inline-module generate Acme::Math::XS::Inline

and we assumed authors would commit this code, and regenerate it from time to time as Inline::Module required it to change.

(NOTE: This is all about author side experience. The user side installation process and result remains the same.)

Last week we called BS on that:

  • Generated code should be automagic
  • Generated code should not get committed if possible

We think we figured out how to make this work in a few styles that will fit in with various module author development styles. One of our goals is that when you use Inline::Module, you can test using the pure Perl mantra:

prove -lv t/

But at this point it is too late to autostub. We need something to happen just before that. One idea is to do this:

PERL5OPT='-MInline::Module=autostub' prove -lv t/

or:

export PERL5OPT='-MInline::Module=autostub'
... later on ...
prove -lv t/

This adds a CODE ref to @INC to do the autostubbing, just in time.

We'll have a few ways to do it, so that if you are someone who likes to do everything explicitly, you can still just:

perl-inline-module generate Acme::Math::XS::Inline

This autostubbing should be done in the next couple days. We'll let you know how it turned out in the next review.


Inline Grant Weekly Report #5

Refinement and Releases. Those are the two words I (David) would use to describe what went on this week in the Inline Grant Project.

Thanks!

First, we would like to thank those members of the Perl Community who contributed their suggestions, work, and code to the Inline Grant Project. leont++ presented a strong case (in the form of a cohesive code example) of how to clean up our Makefile.PL by moving work out of a FixupMakefile function, and into a postamble action. Whereas previously the Makefile.PL might have looked like this:

WriteMakefile(
  …
);

FixupMakefile(
  …
);

Now they look like this:

WriteMakefile(
  …
  postamble => {
    inline => { … },
  },
);

which is a lot cleaner because it works within "the Perl toolchain system" instead of layering an additional step into it. This is documented in Inline::Module::Tutorial.

We also are receiving help from ether++ on updating our Dist::Zilla::Plugin::InlineModule plugin to work within this cleaner framework.

Additionally, we released Module::Install::InlineModule; a plugin to bring Inline support to Module::Install based distributions. So for those keeping score, we now support ExtUtils::MakeMaker, Dist::Zilla, Zilla::Dist, and Module::Install based distributions for Inline.

Modules, Modules, Modules

We have also released the amazing Alt::Acme::Math::XS::EUMM in a new Module::Install based version. Once again for the benefit of those keeping score, the Inline Grant Project has, to date, spawned eight new modules; some plugins, some proofs of concept, and of course Inline::Module itself. Additionally, it has directly contributed to new releases of Inline, Inline::CPP, Inline::C, and many other support modules.

This next week we intend to finish revamping the Dist::Zilla plugin, tackle Module::Build support, add more tests, and finally convert a couple of pre-existing CPAN modules to Inline-based modules.

It's gratifying to see the level of enthusiastic support this project has attracted from the Perl Community. We're seeing more familiar faces in irc.perl.org#inline, receiving more suggestions, and benefiting more from code contributions than ever before.

As we move across the inchstones leading to the wrapping up of the Inline grant project, we are encouraged by the support that we've seen, which demonstrates to us that the end of the grant will only be the beginning of the growth and revitalization of the Inline project.


Inline Grant Weekly Report #4

The major events of this week were:

Inline::Module::Tutorial

This is a work in progress. It explains how to make an XS module using Inline in very basic, specific terms. It only details what we have implemented so far, although there are stub sections for upcoming things. Please give it a read and let us know what you think (in the form of Issues and/or Pull Requests on https://github.com/ingydotnet/inline-module-pm).

Dist::Zilla Support

This is a fun story.

I(ingy) thought I knew how the Dist::Zilla stuff was going to work… I'd simply add these lines to a dist.ini:

[InlineModule]
module = Acme::Math::XS
inline = Acme::Math::XS::Inline
ilsm = Inline::C

then write a Dist::Zilla::Plugin::InlineModule and make it all work. In the end that's exactly what happened, although the path was bumpier than expected.

I started by strolling over to #distzilla on IRC and asking ether++ a few dzil related questions. Then leont++ jumped in and wanted to know what was going on. I told him a bit and he thought I was getting some things wrong… So I invited him to start coding with me (PairUp style)!

It took us a few minutes to get on the same page, but eventually we did. We got some basics into the plugin but then had to break for other stuff. When I came back, I got ether to PairUp for a while. Pretty soon we had a new module, just barely working well enough to release and then release the Acme module based off of it.

TeamProgramming++! Thanks ether++ and leont++!

Zilla::Dist Support

Once we got DZ supported, supporting ZD was trivial. The only difference is where the data goes. In this case it goes in the Meta file (where all the meta data does):

=zild:
  inline:
    module: Acme::Math::XS
    inline: Acme::Math::XS::Inline
    ilsm: Inline::C

Of course, Zilla::Dist uses that info to generate the same dist.ini section above at make release time.

Talking About My Generation

It's a good time to pause, and realize that the whole of programming is pretty much just the generation of one thing from another. Programmers might not think about it that way all the time, but I(ngy) tend to only think about it that way.

When I get frustrated by the ways that I need to manage text to make the computer understand my idea, I often step back and think "What is the world I want to be in right now? How would I rather be doing this?". If I come up with a killer answer that works wonderfully on almost all levels, I make it so.

I'm not sure what drives other folks to make new modules and languages and tools, but that's a big part of my MO. Inline is like that and Inline::Module is too. What is the minimal info I need to tell the Perl toolchain to Make It So?

</opinion>

The Acme Example Modules

We've rearranged the Acme::Math::XS git repo a bit. See this for details: https://github.com/ingydotnet/acme-math-xs-pm/blob/master/About.pod

We've made Acme::Math::XS::XS become Acme::Math::XS (and deprecated the former). This is the pure (no Inline) version. It s on the xs branch of the repo. All the Inline::Module versions will be released to CPAN using the Alt names listed in the link above.

It is important to release this code as different modules, even though the code is exactly the same. We want to have proof that the modifications to the various build systems actually work in the wild.

Next Week

The next big step is definitely writing tests. Doing this type of work Test First would have caused too much churn early on. We needed to experiment and invent a non-trivial process that didn't exist. Now that we see it clearly, writing tests (and TDD) is the right choice.

We now have real, usable, open source software on GitHub, so the roadmap and problems will start being expressed as Issues: https://github.com/ingydotnet/inline-module-pm/issues

Finally, we need to convert some real XS modules like:

to use Inline::Module. This will happen first under the Alt namespace, then once they pass all tests same as the originals, we can make the real ones use Inline::Module.

I encourage interested people to do the same (but maybe wait until next week to start :).


Inline Grant Weekly Report #3

David and Ingy try to get the weekly report out by Saturday night each week, in order to make the Perl Weekly cut. Last week and this week, Saturday drifted into Sunday. Last week was due to unexpected visitors and this week Ingy was wiped out after his Pittsburgh Perl Workshop talk:

https://www.youtube.com/watch?v=vDRLIjojlhg

We had several great days of hacking this week. The goal was to get a fully working Acme::Math::XS, with these properties:

  • Implemented with the new Inline::Module
  • Plain old Makefile.PL
  • Perl module with C code "Inline"
  • perl Makefile.PL && make test works author side
  • prove -lv t/ works (the same) author side
  • make dist builds a dist that:
    • Includes Inline::Module build support code (in inc/)
    • Can build something to install user side, that:
      • Looks just like a normal XS install
      • Doesn't need to install Inline

This is consistent with what we said we'd do in last week's report. We got this 90% done, and released a new Acme::Math::XS.

The next steps are:

  • Finish up Inline::Module good enough for a perfect Acme::Math::XS
  • Write tests for the various concerns and expectations
  • Write up an Inline::Module tutorial
  • Port different kinds of existing XS modules to Inline::Module
  • Inline::Module support for common dist frameworks:
    • Dizt::Zilla
    • Module::Build
    • etc
  • Start filing bugs and nits as GitHub issues on Inline::Module

After that, the grant goals will be mostly satisfied, and we will finish this one up!


Inline Grant Weekly Report #2

This week was a little bit slower but we made progress. We released new versions of Acme::Math::XS and created/released Acme::Math::XS::XS. The two modules are the same except the former uses Inline::Module and the latter uses plain old XS. The idea is to get them both working identically so that there is something concrete to compare.

We did a bit of yak shaving early in the week. We wrote our own blog site software that we are growing up organically. We also patched and released Swim.pm, the markup formatter that all our writings are written in. We are trying to balance staying focused on finishing the grant on time, and also spawning as much cool stuff as we can along the way.

Speaking of cool, we do all our work in a shared dev environment called PairUp™. It works wonderfully, but we wanted others to be able to watch along too (televised pair programming). The way to do this is with termcasting which shows the tmux session in a webpage, so anyone can watch along. Combine this with IRC, and it gets awesome. Unfortunately our termcasting setup was failing us. Fortunately we found doy++ on IRC, brought him into our pairup, had him fix things (he's the master of termcasting) and then we got it working. Come by #inline sometime and watch for yourself.

We further honed the Inline::Module dance. I'll try to explain it. When you create a module like Foo::XS, the lib/Foo/XS.pm module has a line like this:

use Foo::XS::Inline C => "…";

That means you need a lib/Foo/XS/Inline.pm, and you get that with:

> perl-inline-module create Foo::XS::Inline

The new, generated module looks (more or less) like this:

package Foo::XS::Inline;
use base 'Inline';
use Inline::Module 'v1' => '0.02';

When you run tests, this module uses Inline::C to build the C under blib/ like normal XS.

Now, here's the trick (and we haven't actually gotten it working yet)… when it's time build the dist we rearrange things:

  1. The version of ::Inline above goes into the inc/ directory
  2. The ::Inline under lib becomes a small Dynaloader invoking module
    • This is what gets installed user side
  3. The Makefile system is tweaked to build the Inline code at make time

We should get this working in the next couple days. Note that the whole point here is to make something that:

  • Works as good or better than hand-XS
  • Is easy to distribute using any popular setup
  • Installs stuff identically to XS
  • Has no user dependencies (including no dep on anything Inline)

In other news, David released Inline::CPP five times and got his cpantester's PASS rate up to an all-time high of 99.4%.


Inline Grant Weekly Report #1

We (David and Ingy) had a productive week bootstrapping the Inline-for-XS-Modules project. Our primary optimization flags are:

-Opublic
-Oagile
-Ocreative
-Ofun

Here are some highlights:

  • All work being done publicly:
    • David and Ingy code in remote PairUp™ session (tmux)
    • All realtime communication in IRC (irc.perl.org#inline)
    • Tmux to be termcasted soon (live terminal in a browser)
    • If you are interested, come join us!
  • Figured out a decent Inline Module API strategy
    • Authors do 3 simple new things to ship Inline code to CPAN
    • No end-user dependency on Inline for these modules
    • End result is essentially same as hand written XS
  • Plans to support:
    • 5-6 build systems: ie Dist::Zilla, Module::Build, etc
    • Many Inline languages (other than C/C++ might work)
    • Different extension module use cases: inline/external/libraries
  • Created specification document: http://inline.ouistreet.com/node/v3e7.html
  • Released two new CPAN modules:
    • Inline::Module (The majority of the grant coding)
    • Acme::Math::XS (first test module using Inline::C)
  • Created a new blog site to document the Inline Grant Project
    • http://inline.ouistreet.com/
    • A new site using some of our other loved technologies: Cog, Coffee, make, Swim, Jemplate, gh-pages, BootStrap

It is our hope to finish out most of the work in the next 2 weeks, and then work on bug reports after that. We plan to keep putting out blog posts/reports for at least a couple months after the grant is complete. We believe this work will make Extension Module Authoring a lot more accessible, and we want to keep the energy flowing.

Here's a tiny example of how to make an Inline Extension Module. Take this one-liner:

perl -E 'use Inline C=>"int add(int a, int b){return a+b;}";say add(2,2)'

Turn it into this module:

package Acme::Math::XS;
our $VERSION = '0.0.1';
use Acme::Math::XS::Inline C => "int add(int a, int b) {return a + b;}";

Install Inline::Module:

cpanm Inline::Module

Run this command line:

> perl-inline-module create Acme::Math::XS::Inline

Add this line to your dist.ini:

[InlineModule]

Ship it:

dzil release

One advantage we've seen so far is that simple developer testing, just works:

prove -l t/

There's no need to run make to build the C/C++/XS in blib and then add -b to the prove flags, because this is Inline!


Inline Grant Accepted

(or How I Learned to Stop Worrying and Love The Perl)

Last week I returned from 3 weeks of adventure in Berlin and Netherland. Funny story… the night I arrived at Liz and Wendy's place, I was made aware of 2 things:

  • Liz is on the TPF grant board
  • She had a surprise for me, but couldn't tell me until midnight :)

Spoiler: The Inline Grant was approved!

The following evening, I attended the amsterdamx.pm (expats) at Booking.com++ HQ, and upon entering the meeting room, was asked by SawyerX if I could give a talk since ribasushi was stuck in traffic. Huh? WTF? Definitely! :) In the kind of coincidence that only happens regularly in the world of Perl, Karen Pauley happened to be there as well, not to mention Stevan, Jarrko and Dan Kogai! I rambled on about a dozen things that were currently loaded into ingy-RAM, but held off on talking about the grant as it hadn't been made public just yet.

7 days to the minute later I gave a talk at amsterdam.pm and the main subject was the Inline grant. Instead of a talk, I gave a "listen"! I asked everyone there how they would go about actually implementing the grant solution. I got a lot of ideas, and since then I think I've figured it out. Stay tuned…

Welcome to the Inline Grant blog!

I think this is the first TPF grant to be given to a pair of pair programmers! David and I want to make things as open as we can, so we plan to give weekly updates in addition to the TPF monthly status updates.

Today (October 20th) marks the start of work in earnest on the grant. The delay is due to my unexpected but most fruitful trip to Europe. I'm dejetlagged and ready to hack! David has also just finished a lovely family vacation (to Zion and Bryce National Parks (YAPC::15 attendees with wheels should take note!)).

We really want to thank the community for this opportunity to make Perl better!