/dev/dump

illumos hackathon a resounding success

2011-10-25T08:01:00.000-07:00

Yesterday we had our first ever illumos hack-a-thon. About a dozen people from the community showed up, and worked on some very very cool things. At the end of the day, about a half-dozen different demos were done, to show what was worked on.

We'll be posting photos shortly. But here are some of the projects that got attention:

tab completion for dcmds and data types in mdb
::print for DTrace
expanded truss support for ZFS ioctls (nvlist expansion)
time-ordered output for DTrace
a comment field in the vdev label for ZFS pool devices
the ability to change the GUID of a ZFS pool

Several other projects were in progress.

We selected these projects out of a much larger list of project proposals (which I'll post soon), based on the what people thought was most useful, and the ability to achieve results in a single day hack-a-thon. (And what people we willing to either work on, or mentor.)

Most importantly, people got to work on areas that they weren't intimately familiar with, and work with mentors who were much more familiar. For my own part, I had a blast working with George Wilson to implement the ability to alter the GUID of a pool (which is going to have some further uses we can talk about later). My webrev of these changes is now on-line, so I welcome review feedback.

We had domain experts like Adam Leventhal, Matt Ahrens, Eric Schrock, George Wilson, Gordon Ross, and Dan McDonald on hand, to help mentor projects if they needed it, and the amount of cross fertilization was amazing.

While none of the actual code changes are absolutely earth-shattering, they are still very very cool, useful things, that many of us will be happy to have available. I'm already imagining several useful use cases for the ability to reguid a pool, for example. And I can't wait until I have ::print in DTrace and tab completion in mdb. These will make my life significantly better.

Ultimately, this was the most enjoyable coding experience I've had in a long time. I can't wait until we do it again!

NexentaStor 3.1 available now

2011-07-29T17:55:00.000-07:00

After a long, and arduous, release cycle, I am pleased to report that NexentaStor 3.1 is available now.

Customers running existing 3.0 installations may upgrade at no cost.

This release includes a number of key features, including some significant improvements for performance and manageability.

Folks using SCSI target mode will probably see the biggest performance boost relative to earlier editions of NexentaStor, especially those folks using NexentaStor to serve up storage to VMware guests -- thanks to the VAAI offload support that is part of this release. And for the record, yes, this release includes the fix the long standing problem with iSCSI timeouts.

For ZFS fans, this release also includes the updates for ZFS version 28. (This does mean that folks upgrading need to be cautious -- their pools will not be automatically updated to ZFS version 28, but if they are manually updated then there will be no way to move those pools back to an older release. That also means that pools should not be updated in HA cluster configurations unless both cluster partners are updated to NexentaStor 3.1 first.) This also means faster snapshot creation and deletion, and the ability to import pools in read only mode or that have encountered a failure of their SLOG device, making some disaster recovery scenarios much less challenging.

Nexenta Core Platform (our general purpose Debian based open core) will be updated shortly to 3.1 as well -- probably sometime in the next week or so. This will be the same core software as supplied with NexentaStor. (We do strongly encourage folks using Nexenta Core Platform for serving up storage to use our commercial NexentaStor product. There is even a free edition for users with up to 18TB of data.)

We are already (and have been for a while actually) hard at work on the next release, which will be based upon illumos, and include a number of other innovative features. Stay tuned for an update.

Bombproof taskqs

2011-07-28T07:39:00.000-07:00

As part of fixing some recent bugs, I integrated the following into illumos:

734 taskq_dispatch_prealloc() desired

943 zio_interrupt ends up calling taskq_dispatch with TQ_SLEEP

The interesting one is the first of these. The interface is actually called taskq_dispatch_ent(), and is private to the "consolidation" (i.e. for use within bundled code only).

What this interface provides for, however, is a way to bomb-proof your taskq dispatches, if you can arrange for the dispatching state structure (taskq_ent_t) to be allocated in advance. This means you never have to worry about the possibility of a dispatch failing due to insufficient resources.

What's even cooler, is that the cost of the dispatching is much cheaper; taskq_dispatch() was the hottest piece of code on a very busy storage server. Now it goes much much faster, because it is just twiddling some linked list pointers and sending a signal to wake up the thread processing the taskq.

More importantly, the fact that we don't ever have to sleep, or have an expensive call into the kernel memory subsystem, has solved some frustrating "stalls" in the storage subsystem.

I encourage developers working with taskqs in illumos to have a look at this interface. If you can use it, you will simplify your code, and shorten your run-paths. Both of which are good things. The only limitation: this interface is not available for dynamic taskqs. (Which makes sense since dynamic taskqs might need to allocate whole threads.)

illumos podcast

2011-06-13T21:01:00.000-07:00

Constantin Gonzalez recently interviewed me for his "HELDENFfunk" podcast series, while I was in Amsterdam for our European User's Conference. Also interviewed were OpenIndiana founders Alasdair Lumsden and Andrzej Szeszo.

illumos Panel Discussion in SF Bay Area

2011-06-13T20:57:00.001-07:00

All:

There will be a panel discussion about illumos as part of the San Fransisco and Silicon Valley OpenSolaris User's Group meeting tomorrow, June 15. I will be joining Bryan Cantrill and Adam Leventhal to answer your questions about illumos.

The doors open at 6:45pm, and we will continue until about 8pm.

The location is 275 Middlefield #50, Menlo Park, California. Hope to see you there!

NexentaStor 3.0.5 available now

2011-05-02T14:42:00.000-07:00

NexentaStor 3.0.5 is now available.

Apart from fixing some key bugs, the main thing that this release includes is a significant update to the CIFS stack, which addresses both performance concerns, and AD failover concerns.

Note that NS 3.1 is due out *any day now*, and will include all these changes, plus a boat load of others. I'll have a lot more to say about the 3.1 release soon.

GSoC Candidates Selected

2011-04-28T14:27:00.001-07:00

You may be aware that we have selected two candidates for the slots allocated by Google to illumos -- the first is to replace some system utilities from code in perl to native C. The second of which is to bring GRUB2 to illumos.

What you may not know, is that Nexenta will be sponsoring three additional candidates to pursue projects of their own to benefit illumos. These candidates have been selected already, and we will have more to say about them and their work in the future. Stay tuned!

Thanks again, Joyent!

2011-04-28T14:22:00.000-07:00

Joyent have continued to demonstrate their commitment to and support of illumos.

In addition to a string of recent source code integrations, they are now hosting some of our infrastructure in their cloud, with more to follow.

After moving the stuff there, we're now enjoying significantly better performance, and enhanced functionality. Try out the new OpenGrok instance yourself to see!

I'd also like to give a special thank you to Circonus, who are providing active monitoring services for our site now, as a gratuity to illumos. Apparently, they're going to be hosting their stuff on illumos based systems as well, so there's additional synergy here.

What is this OSUNIX thing anyway?

2011-04-19T11:36:00.001-07:00

So there has been some things brewing in a sub-sect of the illumos community about a project to fork illumos, because of alleged problems with my leadership. You can read the thread here if you want.

I want to address this head on.

First the claim is that I've got omnipotent control over illumos. This is absolutely false. While I created the project, and serve as technical lead, I've offered to step down if the developer-council and admin-council would like to me to do so. Notably my employer (Nexenta) has minority representation on both councils, and I've tried to keep the groups as neutral as possible. I said when I created the illumos project, and I still maintain, illumos is a community project, not a Nexenta one.

I'm working on the process to make this more formal through non-profit governance. I should have more to say here before the end of week. (I've got a meeting about this today.)

I've also handed over determination of the Advocate list (the list of people who get to approve and integrate submissions) to developer-council. So far Nexenta has 75% of the advocate slots, but this can change at the request of developer-council. Since about 75% of the contributions to the illumos code have come from my team at Nexenta, this should hardly be surprising. In fact, I've flatly refused to add any more Nexenta advocates, even though there are meritorious candidates, until we get broader representation here. (Becoming an advocate requires making a number of good, well-formed, contributions. And it requires people willing to perform thorough review.)

There is a claim that I've somehow driven companies away from illumos. I hope not. As of now, I'm not aware of any companies that have requested to participate or contribute, who I've turned away. In fact, the only contributions that have been turned down have been Joerg Schilling's star project (he couldn't find people willing to review the code) and the ksh93 update (which has been unable to pass a technical code review -- ultimately we'll probably take in the ksh93 changes in more piecemeal fashion breaking them apart into reasonable and reviewable integrations instead of a 100KLOC+ set of code of varying quality.) As far as I know, everything else is vaporware.

I'd love to know what companies I've driven away, and what I did to do so. Honestly, if there is constructive criticsm here, then I want to hear it because I want to a better job -- and I want illumos to be as inclusive as possible. The fact that nobody has come forward (and nobody has approached me privately either!) makes me wonder how much this is really happening.

In fact, I have done all I can to encourage contribution, and to give credit for such contributions where it is due. And indeed, we have contributions from Joyent, Areca, and others. And a number of things queued up from names like Intel and LSI.

At the end of the day, if the project forks, so be it. Forks aren't necessarily a bad thing, and if a fork means we get more contributors to the ecosystem, then I welcome it. But I hope that the basis for such a fork is not just because one or two people don't like me.

(For the record, I am perfectly happy that I'm not everyone's favorite person... my job is to do the best I can for the future of the project, not to be the most universally well loved person. The open source world is filled with other personalities who people have strong feelings about -- Linus Torvalds, Richard Stallman, Theo De Raadt, Andrew Tridgell, and I don't think that the projects they lead have suffered any for it.)

Anyway, I hope that explains my position. If someone wants to have an open dialog with me about any of this, I'm happy to do so. I don't monitor the OSUNIX lists normally, but I'm reachable via email, IRC (gdamore), this blog, twitter (gedamore), and the developer list on illumos.org.

Thank you Joyent!

2011-04-16T04:04:00.000-07:00

Joyent posted an update -- they've released a branch of illumos, on github, containing much of their illumos contributions.

Some of the stuff is probably fairly Joyent specific, but some of it is highly useful to almost everyone using illumos!

From their mail:

ZFS I/O fair-share scheduling for zones
the Joyent brand, which can be used as a template for other non-SysVR4 or IPS zone brands
Reintroduction of sparse zone images
Crossbow vnics on demand for zones & non-unique vnic naming (unique per zone, not per system)
svcs enhancements ( svcs -Z/-z for interrogating zone services, -L for outputting log files directly (no more ls /var/svc/log | grep... ))
vfsstat and iostat tweaks and ziostat, iostat(1M) for ZFS I/O
more per-zone IO kstats
the zonemon utility for zone kernel state troubleshooting
DTrace enhancements such as llquantize

I just want to say again, thank you very much Joyent! Now, how quickly can we merge this stuff into illumos mainline?

CFV: illumos content authors

2011-04-07T02:10:00.000-07:00

I'm looking for people interested in contributing content to the illumos website. Right now we have a test website but it needs help with producing content. First and foremost we need English content, but the new framework will support other localizations as well.

If you're interested in contributing here, drop me an email. I'll be setting up a mailing list for this soon.

Thank you Areca! 6Gbps 1880 support in illumos

2011-03-30T18:32:00.000-07:00

A big thank you goes out to Areca.

Areca have provided the source code for their Solaris driver, including support for the newer 6G RAID adapters. As a result, I've integrated a (somewhat cleaned up) copy this code as an update to arcmsr(7d) in illumos, under generous open source licensing:

changeset:   13305:fb26af81b9b2
tag:         tip
user:        Garrett D'Amore 
date:        Fri Mar 25 22:14:56 2011 -0700
description:
      834 need support for Areca 1880 6Gbps
      Reviewed by: Dan McDonald <danmcd@nexenta.com>
      Reviewed by: Albert Lee <trisk@nexenta.com>
      Reviewed by: Richard Lowe <richlowe@richlowe.net>

This will make another HBA option available to folks. (Note these cards do support a JBOD mode, so you don't have to use hardware RAID -- indeed I would recommend that you don't when you have ZFS on the disks.

Another outlet..

2011-03-28T15:23:00.001-07:00

So, at the recommendations of others, I'm on twitter now.. Don't know how often I'll keep it updated, but I'll try.

illumos has Serbian Family Language Support

2011-03-23T12:45:00.000-07:00

I just integrated:

changeset: 13312:537259ad27f6
tag: tip
user: Garrett D'Amore
date: Wed Mar 23 08:35:14 2011 -0700
description:
324 need serbian locale support
Reviewed by: Rich Lowe
Approved by: Garrett D'Amore

This is a bit unusual relative to most of the locales, because Serbo-Croatian is a language fraught with some unique political considerations:

There is a common root language, that everyone speaks and understands. But speakers of it rarely agree on what to call it. In Serbia its Serbian. In Bosnia its Bosnian. And so on for Croatian and Montenegrin.

In illumos, we have followed the Unicode CLDR example, and we now have these locales:

hr_HR.UTF-8 - Croatian in Croatia

sr_BA.UTF-8 - Serbian in Bosnia and Herzegovina

sr_ME.UTF-8 - Serbian in Montenegro

sr_RS.UTF-8 - Serbian in Serbia

I want to apologize to anyone offended by this decision, but rather than make a contentious decision on our own, I decided it was best to simply follow the decisions of an international standards body. I believe that there is no fundamental difference in the languages, although some national variances appear to be present in the data files. If someone has more accurate names for these, or believes that some aliased locales will assist with compatibility with other operating systems, then I would be happy to hear suggestions. Ideally from someone familiar with accepted practice in these locations.

There is another wrinkle in all this too. This language -- thanks largely to occupation by Soviet forces as part of the SFR Yugoslavia, is commonly represented using two different alphabets -- Cyrillic and Latin. Generally most locations use Latin, but within Serbia, Cyrillic is mandated by law. So sr_RS uses Cyrillic, while the others use Latin.

Here are the two alphabets:

А Б В Г Д Ђ Е Ж З И Ј К Л Љ М Н Њ О П Р С Т Ћ У Ф Х Ц Ч Џ Ш

A B C Č Ć D Dž Đ E F G H I J K L Lj M N Nj O P R S Š T U V Z Ž

Anyway, if someone sees room for corrections or improvements here, especially if they are familiar with the language(s) and/or region(s), I would appreciate hearing back from you.

Planet OpenSolaris isn't

2011-03-19T16:53:00.000-07:00

It would appear that the old Planet OpenSolaris is no longer a community site.

At least the only blog posts that seem to be there anymore are those that are hosted on blogs.sun.com.

Certainly my posts, which used to show up there until quite recently, no longer do so.

Its possible that this is just a technical snafu, but the recent burst of posts there from Oracle employees suggest a shuffling of things internally in how Oracle handles blogs, and I suspect that eradication of community posts is just one more step along the way.

Of course, if I'm wrong, this post will show up there, and I'll have egg all over my face. :-)

Update: illumos is accepted as GSoC Mentoring Org

2011-03-18T12:39:00.000-07:00

Great news! We (illumos) have been accepted as a Google Summer of Code mentoring organization.

If you are a student and want to get paid this summer to work on an enterprise grade operating system, please have a gander at our ideas page, and then go ahead and start an application.

You can find our application template on our organization information page on the GSoC 2011 site.

Good luck to all the applicants!

illumos gets documentation!

2011-03-14T13:57:00.000-07:00

With this integration:


changeset:   13304:b54231762cfa
tag:         tip
user:        Richard Lowe 
date:        Mon Mar 14 14:05:30 2011 -0400

description:
        243 system manual pages should live with the software
        Reviewed by: garrett@nexenta.com
        Reviewed by: gwr@nexenta.com
        Reviewed by: trisk@opensolaris.org
        Approved by: gwr@nexenta.com

We now have manual pages in illumos. (Only the English pages -- POSIX locale -- are kept in the illumos code repository.)

This is key because it means that code and documentation can be maintained together, which is how some other projects (but not Solaris) manage it.

So, got a problem with the man(1) pages on illumos? File a bug! There is a category called "manpage"... please let us know, or even better, contribute a fix!

Google Summer of Code & illumos

2011-03-06T08:33:00.000-08:00

Got a pet project for illumos you would like someone to take up, and would do yourself if you had time? Like working with bright up and coming stars?

Are you a student looking to get involved with a nascent community of world class engineers, and have some free time on your hands this summer?

Maybe you can participate in Google's Summer of Code. We hope illumos will be selected to participate this. Some ideas are posted on our wiki already, but I'd love to hear other proposals. We have a very short window of time before we have to submit our mentoring org application, so let us know!

COMSTAR and SCSI UNMAP

2011-03-04T14:08:00.000-08:00

I just pushed this for Dan McDonald into illumos:


changeset:   13297:4b9dc4ca8e9f
tag:         tip
user:        Dan McDonald <danmcd@nexenta.com>
date:        Fri Mar 04 13:57:09 2011 -0800
description:
701 UNMAP support for COMSTAR
Reviewed by: Garrett D'Amore <garrett@nexenta.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: George Wilson <gwilson@zfsmail.com>
Approved by: Garrett D'Amore <garrett@nexenta.com>

This change represents a significant new feature in COMSTAR and ZFS, which will greatly benefit people use SCSI target mode functionality in situations involving over-provisioning. More on that in a minute...

The feature itself was developed by Sumit Gupta for Nexenta, and is part of our upcoming 3.1 release of NexentaStor.

Subsequently, Dan took ownership of that code, and working with Eric and George (who are well established ZFS gurus and had significant and useful feedback) improved it still further, and got the code into illumos proper. I believe this represents the first significant ZFS feature to go into the tree since the illumos fork, and also amply demonstrates the collaboration in the illumos community.

I'm looking forward to more collaboration like this in the community.

Now, what does this feature give you? Well, if you're using SCSI Target Mode (either via iSCSI or Fibre Channel or FCoE) to serve up storage to systems running NTFS or ext4, you will be able to make better use of your storage.

Traditionally, when a file was deleted from a filesystem, it was mostly a matter of book-keeping in the meta data in the filesystem. There was nothing to note this in the underlying storage.

With newer SSDs, and with COMSTAR, the ability to get back this notification is incredibly useful. SSDs want it to do garbage collection or other optimizations thereby improving performance.

COMSTAR wants it because now when your thinly-provisioned zvol gets the notification, we can return the storage back to the pool. Prior to this change, the zvol could only grow, it could never shrink. Now, we can give storage back to the pool when you delete a file on the initiator. This is huge in environments running with a lot of VMs using thinly provisioned storage with overallocation.

Anyway, this is now in illumos thanks to Nexenta, and notably, Oracle doesn't have it. Of course, they are welcome to pick up the code for it, but they will need to follow the terms of the CDDL if they choose to do so, the same as everyone else.

SCALE illumos Photos

2011-03-04T07:31:00.000-08:00

As promised, I'd send illumos photos from SCALE. Here's the illumos booth staff, from left to right there is Roland, Garrett (your humble author), Delya, Albert, and Rocky. Rocky was there representing Area Data Systems, who are both Nexenta partners and illumos sponsors.

We also have a facebook photo gallery up, which seems somehow apropos since we were right next to the facebook both at SCALE.

I'm also pleased to report that a number of other Nexenta partners were present as well, showing off Nexenta based products. Next year, we hope they'll be back show casing technology based on illumos and NexentaStor 4.0.

Open Source Opportunities near Boston!

2011-02-28T15:55:00.000-08:00

Its now fairly locked into stone that we will be opening an engineering office somewhere not far from Boston (probably just north of it) sometime during 1H2011.

As a result, we've started an aggressive recruiting campaign in the area. I am interested in talking to people with backgrounds in kernel software or device drivers -- especially if the background is on Solaris, but Linux and BSD backgrounds are fine too.

The culture here is startup, and while I'd love to find a few more architect-level candidates, I'm also keen to find folks just beginning their career with a high level of enthusiasm who are talented and driven to become the industry's next generation of storage and networking gurus.

Nexenta itself is still a small company, and so its still a great opportunity to get into the ground floor of what may well prove to be the fastest growing storage company ever. And even better, you can feel good knowing that your contributions will contribute to the greater good -- we are a company committed to Open Source and .. as our motto says, "Enterprise Storage for Everyone."

We'll also be hiring Quality Engineers in the same office, so if breaking stuff is more up your alley than building stuff, then there may also be a place for you.

If this sounds interesting to you, please let me know.

Note that we do not work with recruiters -- individual candidates only please.

illumos at SCALE

2011-02-26T19:23:00.001-08:00

So, I've spent the past day or so here at the Hilton LAX with Albert, Roland, and Delya manning the illumos booth at SCALE. We're at booth #72. We've been handing out disks with OpenIndiana, Nexenta Core Platform (4.0 alpha release based on illumos), and NexentaStor 3.1 (alpha release) as well as the first ever illumos T-shirts for those folks who install one of these on their system (either physical or virtual.) I'll have photos to post soon -- sorry they're not ready yet.

The show has been amazing, and we've made a lot of new connections, both with other members of the open source community, and with new potential users and contributors as well as other potential collaborators.

If you're around tomorrow, please drop by and say hello!

(Oh yeah, and come watch Richard Elling and I mix it up with the BTRFS guys at the open source filesystems panel tomorrow!)

And if you're looking for an open source job, drop by and chat. We are hiring in a number of areas, and I'd love to have a chance to chat with candidates.

See you tomorrow!

New audio driver

2011-01-31T18:10:00.000-08:00

I've just pushed


changeset:   13278:dabee83e3bb7
tag:         tip
user:        Garrett D'Amore 
date:        Mon Jan 31 17:40:15 2011 -0800
description:
    519 RFE audiocmihd
    Reviewed by: gwr@nexenta.com
    Reviewed by: dev@opensound.com
    Reviewed by: trisk@nexenta.com
    Reviewed by: ams@nexenta.com
    Approved by: trisk@nexenta.com

This represents the first significant contribution to illumos by a third party other than Nexenta, and is also the first hardware driver illumos has support for which Solaris does not. This is for ASUS Xonar cards.

You can tell you have a card that could benefit from this if prtconf -vp | grep pci13f6,8788 shows a result.

I expect this will be introduced soon into a forthcoming OpenIndiana build. Enjoy, and a big thanks again to 4Front Technologies for making this contribution!

Welcome to new Nexentians

2011-01-31T17:46:00.000-08:00

I'd like to take a minute to publicly welcome the following engineers, who are well known in the OpenSolaris community, to my team within Nexenta.

Dan McDonald - Dan just started today, and joins us from Oracle (and previously Sun), where he was one of the lead engineers on IPsec and networking security in general. He's also famous as the creator of the internal "punchin" tool used by Sun engineers for many years. At Nexenta he'll be doing some different things, but also will be our go-to man for issues involving the TCP/IP stack within NexentaStor and we expect to see contributions coming from him back into illumos.

Andrew Stormont - Andy started on Jan 1, and is our first software engineer in the UK. He's known for creating the StormOS desktop distribution based on XFCE on top of Nexenta Core Platform. We're looking forward to capitalizing on the synergy of StormOS and Nexenta Core Platform forward.

Roland Mainz - Roland is known throughout the community as "Mr. Ksh93", and previously was the contributor of the single largest integration from the open community into OpenSolaris. We expect he'll be continuing some of the work on improving our userland within illumos, fixing ksh93 bugs, and doing some other interesting things which should result in a direct improvement in measurable quality for both illumos and NexentaStor.

So, please join me in welcoming these three engineers to the Nexenta team, and also recognizing their past and future contributions into the illumos open source community.

Changes to illumos Contribution Process

2011-01-26T18:16:00.000-08:00

First off, let me state that the following changes are aimed at both easing the challenging of contributing changes to illumos, while increasing our level of "confidence" in what changes are being integrated into our source code tree.

Up until now, illumos has used a contribution model that is primarily derived from the model used within Sun and Oracle for Solaris development.

This development model is based on the notion that all contributors have (or had at least) the direct ability to "push" code to the repository, after a certain number of review steps had been followed.

This model works well with a small team, or where all contributors are reasonably well trusted. This is also not typical at all of the way most FOSS projects work. (Indeed, with OpenSolaris, this model was not used for external contribution.)

Going forward, we want to enable a much wider group of developers, some of whom may not hang around long enough in our community to get a high level of "trust".

We also want to enable a contribution process that is more similar to what other FOSS projects use.

So, to this end, we are going to move from "developer push" to "advocate pull". "Advocates" are just our version of "maintainers" or "gatekeepers". (The Linux equivalent of this is Linus' "lieutenants".)

So now, rather than developers pushing changes directly to our mercurial tree, going forward Advocates will take patches from Contributors (either via hg export or patch file), verify that the content of the patch is what was reviewed, and will then be responsible for integrating those changes into our shared master.

Note that as part of this process, the Advocate will be ensuring that the original Contributor is still credited in the SCM change history. So Contributors still get credit for their work.

Also, we will still be insisting on other parts of the contribution process that we already have, such as code review, testing, and verification of legal right to receive the contribution.

The main implication for Contributors is that they can supply changes in the form of regular patches, which frees them from having to deal directly with one SCM or the other (more on that below). The other

implication is that if a merge conflict occurs that the Advocate can reject a change and ask the contributor to resolve the conflict and resubmit.

Note that this whole process is much much more similar to the process used by other big name open source projects, such as Linux.

At this point, I'd like to point out that we have a "clone" of illumos-gate on github. So you can use git if you want to. We also have an hg clone at bitbucket.

For now, Advocates use hg as our "master" repository, but we are also talking about a conversion to git to make life better. That's a more detailed topic of conversation, but mostly the concern about whether we are using git or hg should be irrelevant to contributors, as they can use either and are not directly exposed to the integration step (hg push or git push for example.)

The final tidbit here is that we need to set up a public page with a list of Advocates, but for now the list of illumos-gate Advocates is:

Garrett D'Amore
Albert Lee
Rich Lowe
Gordon Ross

As more people contribute and demonstrate a level of throughness and trustworthiness, I hope the above list will expand somewhat.

illumos sysad/integrator position (NYC)

2011-01-21T14:55:00.001-08:00

I have an illumos partner that is interested in finding a strong system administrator/system integrator candidate for work on an illumos-based product in New York. Candidates need to be strong with Solaris, security, scripting (perl, python and/or ruby, etc.) and should have cross platform experience. If this sounds interesting to you, please contact me directly.

need C programmers/interns in So. Cal.

2011-01-14T22:28:00.000-08:00

I'm looking to hire a few really smart folks who can work in southwest Riverside County, CA (near Temecula, CA). I want people who are sharp C programmers, want to work on open source kernel software, and are eager to stretch themselves. This is a great learning opportunity. If you know anyone like that, let me know!

illumos update in LA

2011-01-06T09:36:00.001-08:00

I'll be giving an update on illumos, as part of my talk in LA this evening. If you're around, please join us!

beadm provided in "C"

2010-12-30T08:10:00.001-08:00

I know this is somewhat controversial for some people, but....

A member of my staff (Alex Stetsenko) has pushed an implementation of "beadm" that is in C. This is actually derived from an earlier C implementation we already had in tree, called "tbeadm", which we already had. So at some level, this consolidation of two different implementations into a single one. As part of this work, the tbeadm version was modernized and improved to provide i18n capabilities and to behave truly as a drop-in replacement for the python version.

As a result of this change, python is no longer needed at runtime by illumos for anything except IPS packaging. Sites and distributions which do not use IPS packaging (most distros don't, actually) no longer need to install python.

Any illumos fans near Corinth, MS?

2010-12-21T08:35:00.000-08:00

I drove across country last weekend, and am in Corinth, MS. I'd be happy to go out for a beer and some chat if there are any illumos fans or OSUGs nearby this week.

I sed(1) so!

2010-12-15T22:33:00.000-08:00

I just integrated a new sed(1) port for illumos. This is derived from FreeBSD, but it includes a fix for a race condition, and support for translated messages. (FreeBSD friends, please feel free to include these changes back -- I've not changed the original BSD license.)

The legacy sed is gone.

This new sed should work on all the old sed scripts, but there were a few tricky parts that changed -- if you relied on parsing the output of the "l" command, or on the fact that legacy sed only treated content as a byte stream rather than multibyte characters, you might be affected.

Also, I've run into at least one sed script which was malformed, but mistakenly accepted by old sed, but which the new version doesn't accept (but instead gives you a meaningful error message.)

The biggest features in the new sed code:

support for -i and -I
support for -E to enable EREs
much more helpful error messages ("command garbled" was just not very specific)

Enjoy, and please report any problems in the illumos bug tracking system at http://illumos.org/. Thanks!

Update: Note that sed -i requires an argument (the extension) unlike GNU sed where the argument is optional. We can fix that, although this would make us less compatible with FreeBSD sed. (Specifically, it would make it nigh impossible to specify an "extension" starting with a dash.) If someone cares passionately about this, they should file a bug and bring it up on the developer list -- I am happy either way.

Update on SATA Expanders

2010-12-08T12:32:00.000-08:00

So we've done some more research, largely following up on work done by Richard Elling, and I have an update on the SAS/SATA expander problem. There is at least some good news here.

The problems that we've had in the past with these have centered around "reset storms", where a single reset expands into a great number of resets, and I/O throughput quickly diminishes to zero.

The problem is that when a reset occurs on an expander, it aborts any in-flight operations, and they fail. Unfortunately, the *way* in which they fail is to generate a generic "hardware error". The problem is that the sd(7d) driver's response to this is to ... issue another reset, in a futile effort to hopefully correct things.

Now the problem is that this behavior is also performed, by default, for media errors as well. E.g. if you have a disk that has a bad sector on it. Of course, if your disk is mostly idle, it won't be a problem. But if you have a lot of I/O going on, its going to result mostly in a melt-down.

There is good news though, because of the way LSI's drivers are designed.

The LSI mptsas driver at least (and I suspect mpt as well, though I don't have code to look at it) treats "bus-level" resets and "target-level" resets as the same. Both of them do a reset, which will of course reset the expander.

But we can disable the most pernicious reset in sd with the following line in sd.conf:

allow-bus-device-reset=0;

This will allow bus-wide resets to occur, but it will most specifically disable the reset in response to generic hardware and media errors. The relevant section of code in sd.c is this:


 if ((un->un_reset_retry_count != 0) &&
     (xp->xb_retry_count == un->un_reset_retry_count)) {
  mutex_exit(SD_MUTEX(un));
  /* Do NOT do a RESET_ALL here: too intrusive. (4112858) */
  if (un->un_f_allow_bus_device_reset == TRUE) {

   boolean_t try_resetting_target = B_TRUE;

   /*
    * We need to be able to handle specific ASC when we are
    * handling a KEY_HARDWARE_ERROR. In particular
    * taking the default action of resetting the target may
    * not be the appropriate way to attempt recovery.
    * Resetting a target because of a single LUN failure
    * victimizes all LUNs on that target.
    *
    * This is true for the LSI arrays, if an LSI
    * array controller returns an ASC of 0x84 (LUN Dead) we
    * should trust it.
    */

   if (sense_key == KEY_HARDWARE_ERROR) {
    switch (asc) {
    case 0x84:
     if (SD_IS_LSI(un)) {
      try_resetting_target = B_FALSE;
     }
     break;
    default:
     break;
    }
   }

   if (try_resetting_target == B_TRUE) {
    int reset_retval = 0;
    if (un->un_f_lun_reset_enabled == TRUE) {
     SD_TRACE(SD_LOG_IO_CORE, un,
         "sd_sense_key_medium_or_hardware_"
         "error: issuing RESET_LUN\n");
     reset_retval =
         scsi_reset(SD_ADDRESS(un),
         RESET_LUN);
    }
    if (reset_retval == 0) {
     SD_TRACE(SD_LOG_IO_CORE, un,
         "sd_sense_key_medium_or_hardware_"
         "error: issuing RESET_TARGET\n");
     (void) scsi_reset(SD_ADDRESS(un),
         RESET_TARGET);
    }
   }
  }

The savy folks here might notice that this is a wide setting, which is true. You can set it on a specific instance of sd, which requires more effort. There is also a better way to do this, by setting the reset_retry_count property to zero. However, setting the sd.conf property for that properly is considerably more complex, because of the byzantine syntax that sd uses to set up target-specific property values.

So, I still recommend avoiding these SATA expanders. But if you have no choice, then using this sd.conf tunable may be a reasonable workaround.

At the same time, I'm investigating the possibility of having this disabled by default for all of Nexenta's customers -- and possibly even in illumos. If you're a SCSI expert and have opinions on the matter, please let me know.

Status Update on the illumos Foundation

2010-12-05T23:29:00.000-08:00

I sent this out earlier today:

We are working with the Software Freedom Law Center & Eben Moglen on the creation of the illumos legal entity. Nexenta have enlisted the help of Damien Eastwood (one of the more prominent former Sun lawyers) to help drive this. Jason Yoho at Nexenta is driving this fairly hard as well, so there are now more people than just me pushing this forward as quickly as we can.

We have set a goal that the legal entity (an illumos foundation) should exist with legal presence before the year's end. I'm told that this is an achievable goal.

I'll have more updates on this soon, I expect, but the process is moving forward.

New Advocate - Albert Lee

2010-12-05T23:27:00.000-08:00

I'm pleased to announce the addition of Albert Lee (trisk@nexenta.com, aka Triskelios on IRC) to the list of Advocates who can approve integrations into illumos. Albert has been doing a lot of excellent work on both illumos and OpenIndiana, and I'm happy to expand the set of advocates we have available to include such a diligent and talented individual.

The current list of Advocates for illumos-gate are:

Garrett D'Amore
Albert Lee
Rich Lowe
Gordon Ross

I'm hoping to expand the list to include more non-Nexenta-employees as well. If you're a contributor and would like to help out in this way, let me know. Typically becoming an advocate means you have earned the trust of the rest of the advocates by making several "good" integrations into illumos-gate (4-5 at least usually, although some credit is given for previous integration experience with ON at Sun/Oracle), and have a demonstrated level of thoroughness to help us ensure quality integrations.

Thanks, and again, congratulations and thank you to Albert.

New open source iprb(7D) driver

2010-12-01T14:05:00.000-08:00

For a variety of byzantine reasons, the iprb driver has never been open sourced, even though everyone who's ever actually had anything to do with it agrees that it should be. (I blame the lawyers on this one...)

So I went ahead and reimplemented -- from scratch -- a new iprb driver. I'd certainly appreciate feedback on the code, which you can read in the webrev. I'm hoping to integrate this into illumos later this week.

zfs should not depend on python... and doesn't anymore

2010-12-01T14:00:00.000-08:00

As of the recent integration of a colleague of mine, illumos now has a zfs command that does not depend on python at all.

changeset: 13246:fe5d6e0b0bce
tag: tip
user: Alexander Stetsenko
date: Wed Dec 01 02:30:25 2010 +0300
description:
278 get rid zfs of python and pyzfs dependencies
Reviewed by: gordon.w.ross@gmail.com
Reviewed by: trisk@opensolaris.org
Reviewed by: alexander.r.eremin@gmail.com
Reviewed by: jerry.jelinek@joyent.com
Approved by: garrett@nexenta.com

The zfs command is now entirely a C program. This may make it more friendly for use in other environments or platforms. FreeBSD folks, you might want to incorporate this into your tree. If you do, I'd sure like to know.

Job Opp @ Nexenta: Director of Sustaining/Certifications

2010-11-11T17:14:00.000-08:00

We're looking for a Director of Engineering, to own sustaining (aka bug fixing) and hardware platform certifications (where partners provide a hardware platform and ask us to certify it for use with NexentaStor.)

The job qualifications:

Must be local to the SF bay area (because the hardware lab is here)
Must have strong communication skills
Must be able to deal with stressful situations, and able to "manage" strong personalities
Must have Solaris/OpenSolaris expertise... hands-on kernel work (crash dump analysis, coding, etc.)
Desire experience with Storage protocols and products
Desire expertise with x86 hardware
Desire perl and/or python skills

To be clear, this will start out as a hands-on position, with a fast-paced startup environment. But the growth opportunities here are enormous. If you think you're up to this, please let me know.

New desktop image

2010-11-05T20:29:00.000-07:00

Here's a sample of the new logo as a desktop image. I've not made this available publicly yet (mostly because I don't know how to capture this in a form that will include the gradient and post it where people can find it.) If someone with some gnome expertise on how to share this for others contacts me, I can work to make it available.

New illumos logo

2010-10-27T23:56:00.000-07:00

Today at the OpenStorage Summit 2010, I unveiled the new illumos logo. We will be updating our branding, which also includes a new font, and other elements, over the next few weeks.

There were other updates on illumos that I covered in this talk. I think this was recorded, but I'm not sure right now where it was recorded and how to acces it. I'll be sure to share that when I find out.

CFV: web/HTML/graphics people

2010-10-13T21:11:00.001-07:00

I have an urgent need to rennovate the illumos website. If you'd like to help the project out, and you have got both time and talent, please let me know. A major overhaul of the site is in order, and we need someone willing to dedicate some time on it. There may be some funds available for the right person, but to be clear, illumos can't afford the services of a professional design bureau.

New implementation of printf

2010-10-13T20:49:00.001-07:00

So I finally got tired of waiting for someone else to do a printf(1) replacement in illumos for the closed binary from Oracle. I had thought this would be a trivial thing to do via ksh93/libcmd using a symbolic link ala /usr/bin/alias.

Lo and behold, it wasn't! Why? Because ksh93 printf insists (like all ksh93 builtins) on having -- and - getopt style processing. This is fundamentally incompatible with legacy printf. (Why does it do this? So it can dump its builtin man page, e.g. printf --man, to the console. A feature I've railed against in the past.)

Here's what should happen:


% printf -v
-v%

Here's what ksh93 does:


garrett@thinkpad:~$ printf -v                                                   
ksh93: printf: -v: unknown option
Usage: printf [ options ] format [string ...]

Now there is an argument to be made that a script which relies on the legacy behavior is fundamentally broken. But it doesn't matter -- the scripts are in the field (there are real examples of them), and the legacy behavior must be preserved. Breaking these legacy scripts just so that we can dump printf --version output is... silly. This is case where pragmatism wins over purity.

Rather than try to rip this out and fight with the ksh93 about "deviation from the upstream" (apparently the ksh93 folks view any changes we make in illumos or OpenSolaris as automatically toxic unless they originate from David Korn or Glenn Fowler), I've just gone ahead and implemented my own printf(1) on top of FreeBSD's. This will be the implementation in illumos.

I've added significantly to FreeBSD's code though. Specifically, I added handling of %n$ processing to get parameterized position handling. This is needed for internationalization -- it allows you to change the order of output as part of the output from something like gettext(1). (This is needed when you have to change word order to accommodate different natural language grammars.)

So my implementation is superior to FreeBSD's, and its superior to the legacy closed binary version. Why? Because rather than a half-hearted attempt at processing positional parameters, my version really handles these, including full support for the usual format specifiers. For example:

New open code:


garrett@thinkpad{4}> printf '%2$1d %1$s\n' one 2 three 4
2 one
4 three

Old closed code:


garrett@master{22}> printf '%2$1d %1$s\n' one 2 three 4
134511600 one

Clearly the old behavior is just plain wrong. For the record, ksh93 does the right thing here too. (Although somewhat older versions of ksh93 would dump core on this command line.)

My diffs (which also include style and lint fixes required for illumos) relative to FreeBSD are online. You can also review a webrev of the changes that I hope to integrate into illumos. The license remains BSD, so the various BSD operating systems (or even Oracle) are free to incorporate these improvements if they like.

illumos gets global

2010-10-08T13:32:00.000-07:00

I just pushed a major set of changes:

8 libc locale work needs updated license files
223 libc needs multibyte locale support for collation
225 libc locale binary files should be in native byte order
309 populate initial locales for illumos

As a result, illumos has gained base support for some 157 different locales, spanning 67 languages and 116 different territories. This includes nearly all the major languages of the world -- missing are Serbian, Javanese, Farsi, Malaysian, Burmese, and some languages spoken in central and west Africa. (Some of these will be very easy for someone else to add... let me know if you want one of these and are willing to do the work.)

The support for these locales includes full POSIX compliant collating support, which was completely absent in illumos before this integration.

Also, included, is a new open source implementation of localedef(1). To my knowledge, this new implementation is the only non-GNU version of localedef that is fully open, and this version is more fully functional than the GNU version. (The GNU localedef lacks full support for collation data.)

Other notes: this is only the base support for these locales. This will for example give localized output from "date". There is quite a lot of additional effort required to fully localize an illumos system, including support for input methods, fonts, and message catalogs for all the various applications. However, with this base support, it makes doing that other work much more practical.

This integration adds nearly 2 million lines to illumos, although far and away the vast majority of it is in the form of data from Unicode and the CLDR (common locale data repository). The ability to import data directly from these sources is the new code that I've written, including a major overhaul of the underlying ctype and collation support in libc to properly support multibyte locales.

Its my belief that with this integration, one of the biggest feature gaps between illumos and Solaris is closed.

Emacs & Gnome Terminal Co-existence Resolved

2010-10-03T23:27:00.000-07:00

For many years, I've been stuck with old xterm, because it was the only one that honored my Meta keys in the same way that GNU emacs did. I could never figure out how to make gnome-terminal work, which always bothered me somewhat. (Notably GNOME terminal has better Unicode support which has lately become important to me.)

I finally found a reference that helped me out. I understood that the problem was conflicting ideas about modifier keys; gnome-terminal uses Mod1, but Emacs uses Mod4. What I didn't know was something I found out here, namely that Emacs only uses Mod4 if it exists. So a better solution for me is to simply clear Mod4 altogether, and both programs happily honor Mod1. (This leaves xterm hosed, but if gnome-terminal works, then I don't need xterm anymore.)

My resulting .xmodmap looks like this:

remove Lock = Caps_Lock
keysym Caps_Lock = Control_L
add Control = Control_L
clear Mod4

This makes my PC keyboard behave sensibly. Alt is Meta. And Caps Lock is consigned to oblivion and the large key that used to have that function is now much more usefully assigned to Control.

I'm posting this here in case anyone else has struggled with this particular annoyance in the past. The clear Mod4 trick was the surprise ticket. (What I'd really like is a way to tell programs which Modifier is "really" the Meta key, given that the programs can't seem to agree on this. And with just one preference -- redefining the numerous bindings in emacs for each sequence, while possible, is not my idea of a fun thing to do.)

The other thing I'd like is a standard way in illumos/opensolaris to integrate .xmodmap. Linux/Ubuntu seems to detect my .xmodmap and handles it nicely.

Another ZFS departure

2010-09-28T09:23:00.000-07:00

Jeff Bonwick is leaving Oracle.

This is a huge event, because Jeff has been one of the main innovators in operating system technology during his tenure at Sun. While you may know him best for ZFS, he's also the inventor of the slab allocator, which revolutionized memory management when it was created. (And now, pretty much every modern system uses some variation of the slab allocator.)

And he's not just an Oracle VP. Jeff has made integrations into Solaris' ZFS code base on an ongoing basis. This is a guy that has led with actual actions and innovation, backed by code, rather than some boffin who's risen to management and no longer contributes. At some level, he's the model for the kind of technologist I aspire to be.

With so many innovators leaving (and yes, there are other key players in flight), its going to be very interesting to see how Oracle is able to continue to be a thought leader in the OS technology that they've acquired.

One the one hand, its really a shame to see to much of the heart and soul of the Solaris engineer core slowly disintegrating.

On the other hand, I think illumos may be the place where Solaris innovation happens, more so than at Oracle, even sooner than I previously expected.

South/Central American opportunity

2010-09-25T16:46:00.000-07:00

I just learned that a peer of mine is looking to add some escalation engineers in Latin America. Job requirements include excellent English, and the ability to deep dive into customer problems including kernel crash dump analysis and C coding ability. If this sounds interesting to you, please let me know.

Oracle/NetApp ZFS lawsuit dismissed

2010-09-09T11:20:00.000-07:00

Others have no doubt already picked upon this, but here it is anyway:

http://www.h-online.com/open/news/item/NetApp-and-Oracle-lift-ZFS-patent-cloud-1076313.html

Hopefully this is good news for downstream ZFS consumers.

We're Hiring!

2010-09-07T14:48:00.000-07:00

In case you didn't know, a number of companies are hiring illumos talent.

I know of an opening for a USB kernel engineer at one company.

I'm told Joyent is growing like crazy.

And Nexenta is hiring! In fact, here are some of the opportunities we have open at Nexenta:

QA leads. We have two positions for folks with skills and knowledge to design and build, and run, automated testing of the operating system, with a particular focus on storage and networking. Expertise in NFS, CIFS, iSCSI, ZFS, and the surrounding areas would be highly useful. Good communication skills, shell scripting or perl skills, and an ability to work in the office in Mountain View, are all required. Previous QA leadership preferred.
Support engineers. We need support engineers across the globe. People who can answer the phone, and triage problems. Solaris or UNIX experience, ZFS clue, good troubleshooting and triage skills, and excellent communication skills are necessary.
Kernel software engineers. I need people with deep TCP/IP, SCSI, Storage, and Filesystems expertise. Solaris expertise highly preferred, but can substitute FreeBSD or Linux kernel expertise. Highly motivated self-driven super-stars only.
Sustaining software engineers. Excellent troubleshooting and kernel expertise is required. Expertise in one or more of TCP/IP, SCSI, storage, and filesystems is preferable. Solaris expertise highly preferred.
IT staff. We have one opening for a mid-level IT engineer. Must be able to deal with Solaris, Linux, Windows, phones, and cantankerous development staff.

I expect even more growth will occur here over time. A jobs board for illumos will be coming soon.

Squash-proof?

2010-09-03T14:09:00.000-07:00

So everyone has heard me talk about the 800 lb. gorilla with respect to illumos.

One question I keep getting asked is, can the illumos project be "squashed" by this 800 lb. gorilla?

My stock answer had been "no". But I realized something today; I've been wrong.

The way illumos can be "killed" is if the corporate owner of Solaris were to do something to make illumos irrelevant. Like, say, opening Solaris back up (and in this case, I think they would probably need to go further open than they were before).

I'm not worried though. Even if that happens, illumos will have been a major success. But I really don't think it is going to happen.

illumos Interest Groups

2010-09-01T12:07:00.000-07:00

So, I've been asked by several people who are involved with OpenSolaris User Groups around the world about illumos.

Given the clear demise of OpenSolaris, it seems to me at least, to be kind of silly to continue to meet using that name.

Some groups have reverted to pure Solaris usage. Which is fine for those groups that want to focus on Oracle products and want to come under the Oracle umbrella that it has for user groups.

For groups that are more interested in open technology, perhaps it is time to start up some "illumos interest groups" (IIGs)? (Calling them "User Groups" at this point seems rather premature... I think there are only a very few of us that are actually "using" illumos at this point.. but I hope that number to grow very much very soon. :-)

Btw, are there any folks interested in illumos in either Riverside County or North San Diego County? (California) I'd be interested in participating in an interest group if there was one that didn't require me to drive over an hour to get to.

OpenSolaris ARC is Dead

2010-09-01T11:49:00.001-07:00

I had tried to dial in to ARC today, but no luck. But then someone else pointed out that we have not seen any ARC cases since the tap was turned off.

In fact, I posted a query about this to the opensolaris-arc mailing list today, and I got back an interesting automated reply:

This mailing list is no longer active and accepting posts. Mailing
list archives can be found at
http://mail.opensolaris.org/pipermail/opensolaris-arc/. You can check
http://mail.opensolaris.org/mailman/listinfo to find another list to
which to send your email.

So, OpenSolaris ARC is dead. This has ramifications that go beyond just ON. Because there are other consolidations that we were promised were going to continue to be developed in the open: JDS, X11, and the pkg-gate. If the decisions for these technologies are no longer being made openly, or even the opinions being made available, then this makes Oracle's promise to continue to work with the community on them seem hollow.

So, what's left for "OpenSolaris" as so named? There are some code drops still being made. How long will that keep up? Are they continuing to take contribution from external parties? (I don't work on those gates, so I don't really know.) I'd like to know if the other consolidations have shut down too. At least the key decisions relating to those consolidations seem to have moved behind closed doors.

OGB has dissolved today

2010-08-23T09:08:00.001-07:00

The old OpenSolaris Governing Board has dissolved unanimously today.

The OpenSolaris governance is now in default, and returns to Oracle's hands.

For folks upset by this, let me remind them of Illumos. Its a sad note for OpenSolaris, but I think the reborn Illumos community will be better than the OpenSolaris community ever could be.

I do want to thank the (former) OGB members for their efforts, even if they did prove to be in vain.

Why SAS->SATA is not such a great idea

2010-08-22T11:49:00.000-07:00

So, we've had some "issue" reports relating to the mpt driver. In almost all cases, the results are related to situations where people are using SATA drives, and hooking them into SAS configurations.

Although the technology is supposed to work, and sometimes it works well, its a bad idea.

Let me elaborate:

SAS drives are generally subjected to more rigorous quality controls. This is the main reason why they cost more. (That and the market will pay more.)
SAS to SATA conversion technologies involve a significant level of protocol conversion. While the electricals may be the same, the protocols are quite different.
Such conversion technology is generally done in hardware, where only the hardware manufacturer has a chance of debugging problems when they occur.
Some of these hardware implementations remove debugging information that would be present in the SATA packet, and just supply "generic" undebuggable data in the SCSI (SAS) error return.
The conversion technology creates another potential point of failure.
Some of these hardware implementations won't be upgradeable, or at least not easily upgradeable, with software.
SATA drives won't have a SCSI GUID (ATA specs don't require it), and so the fabricated GUID (created by the SAS converter) may be different when you move the drive to a different chassis, potentially breaking things that rely on having a stable GUID for the drive.

Don't get me wrong. For many uses, SATA drives are great. They're great when you need low cost storage, and when you are connecting to a system that is purely SATA (such as to an AHCI controller), there is no reason to be concerned.

But building a system that relies upon complex protocol conversion in hardware, just adds another level of complexity. And complexity is evil. (KISS).

So if you want enterprise SAS storage, then go ahead and spring for the extra cost of drives that are natively SAS. Goofing around with the hybrid SAS/SATA options is just penny wise, and pound foolish.

But hey, its your data. I just know that I won't be putting my trusted data in a configuration that is effectively undebuggable.

(Note: the above is my own personal opinion, and should not be construed as an official statement from Nexenta.)

Aug 30, 2010: Update: At a significant account, I can say that we (meaning Nexenta) have verified that SAS/SATA expanders combined with high loads of ZFS activity have proven conclusively to be highly toxic. So, if you're designing an enterprise storage solution, please consider using SAS all the way to the disk drives, and just skip those cheaper SATA options. You may think SATA looks like a bargain, but when your array goes offline during ZFS scrub or resilver operations because the expander is choking on cache sync commands, you'll really wish you had spent the extra cash up front. Really.

IPS == FAIL

2010-08-22T10:02:00.000-07:00

Look, I really, really wanted to avoid entering the packaging debate. I mean, its an emotional decision, right?

Well, its supposed to be.

Except that I've spent nearly an entire day trying to figure out how to onu the latest illumos gate (which includes Rich Lowe's b147 merged in). I have gate changes that I desperately need to test in the context of a full install. (Well, I could say "screw it", and just test the bits in place -- which I've already done, but that's hardly a complete test.) I can't test them. Because I can't figure out how to use the packaging system to install them. And neither can our resident IPS expert, Rich Lowe.

This is no longer an emotional decision for me. Yeah, there are a lot of "emotional" things not to like about IPS. (It forces a dependency upon Python; its still immature; it seems to fail if you are disconnected from the network; it doesn't seem possible to build and install "just" a single package; apparently there are a lot of magic incantations that nobody outside of the IPS developers really understands; etc.) I was willing to set aside all those "emotional" responses and use IPS, if it worked. If for no other reason than the fact that it did away with BFU I have been willing to give it my best effort. But the latest situation has left me dead in the water, and apparently NO ONE can help me.

Look, I'm not a complete moron. (Well, maybe you disagree with me, but this is my blog.)

I should be able to make this work. If I cannot, then what kind of barrier is this going to create for participation from other people? Is Rich Lowe going to hold the hands of everyone else to get past these issues?

What happens the next time the pkg folks introduce another flag day?

This is unacceptable.

I'd like to hear other solutions. At the moment, I'm very very seriously considering gutting the IPS build requirements and having illumos go back to building SVR4 packages natively, using a tool to convert IPS meta data. (So meta data would be IPS, but binary deliverable would be auto-generated SVR4 packages.)

The current situation reminds me of Linus' comments about CVS. I feel the same way about IPS right now. I'm very angry ... the tools that are supposed to facilitate development have caused it to cease for me. If the only way for me to move forward is to reinvent SVR4 build systems, then that's what I'll do.

IPS is a failed science experiment. I don't see how it is going to get widespread adoption from anyone (ISVs or otherwise) with it as it stands today.

Flames to /dev/null. Let me know if you have a solution though.

Update: Rich was finally able to get me to the point of working. Although I can't ever downgrade. After what I just went through, I never want to. I'm really terrified that nobody really understands the steps it took to get me to a working state, and I am unwilling to force others to go through the same nightmare. So I'm still made at IPS, and I still think we need to unhitch the illumos cart from it.

The Tap Is Turned Off

2010-08-19T00:26:00.000-07:00

A little birdie told me that the last update to Oracles hg repository for ON was this one:

changeset: 13149:b23a4dab3d50
tag: tip
user: Sukumar Swaminathan
date: Wed Aug 18 15:52:48 2010 -0600
description:
6973228 Cannot download firmware 2.103.x.x on Emulex FCoE HBAs
6960289 fiber side of emulex cna does not connect to the storage
6950462 Emulex HBA permanently DESTROYED, if the firmware upgrade is interrupted
6964513 COMSTAR - Emulex LP9002 fail to return a SCSI Inquiry correctly to a VMware 4 Initiator

From here on out, Illumos and Oracle Solaris diverge. The funny thing is, based on the calls I've had today, I could hardly be more optimistic about the future of illumos and the code base that was formerly called Solaris. Even more talent is getting behind this effort every day.

I'm very very excited... frankly Oracle shutting down the tap just really opened up the opportunity for us to really start innovating, in ways that I would have been loathe to do if we were still trying to maintain a very closely aligned source tree.

I think its entirely possible that Oracle may wind up viewing Illumos as the upstream rather than the reverse!

More milestones...

2010-08-19T00:25:00.001-07:00

Illumos milestones reached today.

a) I pushed a working tr, and was able to build illumos on a system running illumos. This is the first time this has been possible.

b) Richlowe pushed a merge to build 147. There are probably consequences for developers (more updates required for bits that are not part of ON) -- stay tuned for updates about that.

All in all, things are moving quickly.

Presenting Illumos at SVOSUG

2010-08-17T21:45:00.000-07:00

I'm pleased to announce that I'll be giving a brief talk at this month's SVOSUG meeting, Thursday Aug 26, at 6:45 pm in Mountain View. It will cover Illumos, and I will be joined by a colleague who will talk a bit more about Nexenta as well. If you're in the Bay Area at that time, it would be great to have a chance to meet.

I expect there will be some (probably significant) consumption of alcoholic beverages after the meeting, at an as yet undetermined location.

More new stuff...

2010-08-16T16:35:00.000-07:00

I've been pretty busy with Illumos lately, but last week I took a few days off for family time.

One of the things I did was take my son (9 years old) out to the Kern River to try some whitewater kayaking. This was his first time on moving water, and it amazed me how quickly he picked up basic concepts. He was doing ferries, peel outs, and eddy turns like a champ after about 20-30 minutes. Amazing. He didn't even swim his first day -- he elected to stay in his boat (actually trying to do a roll) until I could give him an Eskimo rescue. (His only swim that day was when he got flipped by one of the holes in Riverside Park.)

He did get a good swim on the second day, when we were working on ferries though the much faster swift water running at the bottom of Ewings rapid. His first ferry was quite high into the rapid itself, and clean, but the second time he went for a swim. Came up happy and smiling, ready to try again if we had had the time.

I wish I had some pictures.

Guess I'm gonna have to get the kid a boat soon. He wants to try kayak surfing with me, and he really wants to learn to roll. Too bad there are no vendors that offer whitewater boats small enough for kids in southern California. We probably won't make it to Kernville again until next season. :-(

Milestone Commit for Illumos

2010-08-15T13:26:00.000-07:00

Richard Lowe has just made a milestone change to the Illumos repository.

Its a milestone for two reasons:

a) It is the first commit from another developer other than me. (Other developers have code in progress, but not yet ready to commit, but soon!) This also makes it truly a community project, since Rich has no affiliation with me other than as a participant in the Illumos project.

b) It eliminates the dependency on the Oracle "extra" repository, which required folks to get a certificate to access non-redistributable code in order to build illumos.

Thank you very much Rich. I'm looking forward to more integrations from developers soon!

The Hand May Be Forced

2010-08-13T18:28:00.000-07:00

Well, as you may have read, Oracle has decided that at some point very soon, we're going to lose normal regular access to the source code for OS/Net. (I.e. the Solaris kernel and supporting programs.)

While I would have vastly preferred for Illumos to have a cooperative and collaborative relationship with Oracle, it appears that Oracle doesn't value this. In fact, the exact words were from the management at Oracle were as follows:

Solaris is not something we outsource to others, it is not the assembly of someone else’s technology, and it is not a sustaining-only product.

While I understand the need to own the technology, there are few things that could be stated that show a stronger NIH attitude than this. Its unlikely that there will ever be a way for Oracle and the greater community to have a collaborative relationship.

This is a dark day for OpenSolaris -- its effectively dead now. (Its parent, Solaris, lives on however.)

How unfortunate.

For Oracle that is.

Because from the fertile ashes of the dead springs forth new life bringing hope and light in the form of Illumos.

Illumos has garnered the support of some of the top minds in the industry; already the list of names of Solaris contributors and potential contributors that have already publicly committed to supporting this project is extensive. Many of the names are famous, people like Bryan Cantrill. Oracle's actions and inaction have actually made this possible.

I can also say, the list goes even further -- considerably so. I have had private conversations with quite a few other people who have quietly committed to involvement. Some of the names are very surprising, and I hope that they will soon be in a position to announce their involvement for themselves. These are people that are big name contributors; folks who have made very large numbers of code commits to Solaris -- some of the deepest and most "challenging" parts of Solaris, too.

The upshot of this is that the future for Illumos is surprisingly bright. Rather than a dependency on the good will of one corporate sponsor with dubious intentions, the project will have the diverse backing of some of the most well-known innovators (and their employers) from the OpenSolaris -- nay, Open Source -- community.

So, by their actions here, Oracle may be forcing Illumos to "fork", which was always a prospect, even if not one I cherished. But with the backing of the innovators I know who are with us, I think we have a chance to actually be the premiere foundation for SunOS derived technology. Oracle may be investing more into Solaris, but if the best and brightest have left for greener pastures and are contributing to Illumos, then I think we'll have the "best" investments in the base. Following Oracle's lead when the brightest minds have already left looks less and less desirable by the moment. (And to be fair, there are still many bright folks within the Solaris organization at Oracle. But the balance is changing, and changing in favor of Illumos and the open development community.)

Oracle Solaris will not be the only source for this technology, and now it appears it may not even be the best source for this technology.

I once said I never intended for Illumos to compete with Solaris. That was true, but if Oracle forces the issue, then even despite their vast economic resources, I say, "Bring it!"

Illumos Announcement

2010-08-03T15:50:00.000-07:00

Today we announced the Illumos Project. I think the call I gave on it had a lot more information than I want to write here, and there are now quite a number of blog postings from other more recognizable names than my own. I'm thrilled by the excitement here!

Illumos

2010-07-30T11:13:00.000-07:00

A number of the community leaders from the OpenSolaris community have been working quietly together on a new effort called Illumos, and we're just about ready to fully disclose our work to, and invite the general participation of, the general public.

We believe that everyone who is interested in OpenSolaris should be interested in what we have to say, and so we invite the entire OpenSolaris community to join us for a presentation on at 1PM EDT on August 3, 2010.

You can find out the full details of how to listen in to our conference, or attend in person (we will be announcing from New York City) by visiting http://www.illumos.org/announce (The final details shall be posted there not later than 1PM EDT Aug 1, 2010.)

We look forward to seeing you there!

- Garrett D'Amore & the rest of the Illumos Cast

Please Be Patient

2010-07-15T08:46:00.000-07:00

With all the ruckus surrounding Oracle's apparent abandonment of the community, and OGB's stated intention to suicide, the community uproar has been crazy.

Without giving any details, let me say that a few of us are quietly but diligently working on solutions to the critical problems, and I expect we'll be able to talk much more freely about the solutions we will be offering in early August, which is coming up very soon now.

So, I'm going to humbly ask folks to be patient -- hold your comments, complaints, and flames about Oracle and OpenSolaris and OGB in check please. If you can wait just a little bit longer, then I believe we'll be able to offer a more constructive outlet for your frustration and energies.

Thanks.

In NYC for DebConf10

2010-07-14T01:57:00.000-07:00

I'll be attending DebConf10 (the Debian developer's conference) in NYC this year. Nexenta will be presenting information about our distribution. Its my hope that we can use this to generate more interest in OpenSolaris technology. If you're in NYC, and want to meet during the first week of August, let me know!

ZFS disk monitoring...

2010-07-07T14:16:00.000-07:00

So I've posted this on zfs-discuss at opensolaris dot org, but its been suggested I mention it here too.

It turns out that the ZFS/FMA integration doesn't pick up on drive removals for most disk devices until the filesystem attempts to perform some I/O to the drive. This is rather unfortunate, because if a file system is not busy, you might suffer a loss of redundancy and not find out about it until too late.

It also means that you won't know about failures of hot spare devices until you need to put them into service, since by definition they are idle. (Note: as an exception running periodic scrubs should detect this too, although scrubs are highly intrusive to the overall I/O load on the system and probably should not be performed too often as a result.)

I'm told the Oracle 7000 series appliances have a solution for this problem, but of course the source for that is not in OpenSolaris. (Apparently there are quite a few differences in the core OS between the 7000 series and vanilla OpenSolaris -- unfortunately we can't know because -- unlike with NexentaStor -- we don't have access to the kernel source tree!)

This is not good for folks who use ZFS with ordinary Solaris 10 or OpenSolaris... or with derivatives such as NexentaStor.

To address that problem, I've developed a some code called "zfs-monitor" that periodically monitors the health of any physical vdev (disk) that is part of a ZFS pool (hot spare, log, or real device). This code is implemented as an FMA module. When a disk goes offline, zfs-monitor detects it, and triggers an FMA event, which allows ZFS to do the right thing. This means if a disk goes away, even if it isn't in use, whatever action is appropriate will be performed. (Logged in FMA fault logs, and if appropriate, a hot spare will be recruited to replace the failed or offline device.)

This code is part of NexentaStor 3.0.3. As there are some semantic differences of opinion (what constitutes device failure versus intentional removal by an administrator), the code is unlikely to be pushed into ON without further change. (At the same time, I've fixed a different problem in the ZFS FMRI parsing code, and I've submitted a request to get that fix integrated -- but I've not heard back from anyone at Oracle who is willing to sponsor the change yet.)

I'm happy to share the code for zfs-monitor to anyone who requests it. (In fact, you can examine the code in our open Mercurial repository directly!) Note that for it to work properly, you also will need the fix for the ZFS FMRI parsing bug just mentioned.

At Nexenta, we're committed to innovating and improving upon the great foundation of ZFS and OpenSolaris, and to the reasonable extent possible, we want to share those innovations with the greater OpenSolaris community. Hopefully changes like this demonstrate this commitment in a tangible fashion.

Looking for CIFS/AD expertise

2010-06-28T22:34:00.001-07:00

(I know its probably questionable using my blog for this, but I thought I'd post it here anyway. My apologies if anyone finds this offensive. I'll keep it brief in any case.)

I'm looking for a high-caliber developer, preferably with some kernel and/or OpenSolaris expertise, who's also got extensive knowledge of ActiveDirectory and CIFS. If that's you, or you know someone who fits that description, please contact me -- garrett at nexenta dot com. (No recruiters or agents please.)

skype for Solaris

2010-06-15T15:40:00.000-07:00

So I'm irked, really irked.

If we had Skype support for Solaris, I could probably ditch this half baked mess of Linux hosts running VMware guests with OpenSolaris and Nexenta. I want just a single host OS for my development box.

Right now the single biggest barrier to running OpenSolaris on my desktop for my job at Nexenta is Skype. But this is silly, because Skype works in Linux, and the APIs should basically be compatible. Especially with the OSS layer that we already have in OpenSolaris these days via Boomer.

If someone at Skype sees this (good luck trying to find a contact on their web site!), and wants to work with me on it, I'd be happy to help them work through the issues of getting a native Skype port.

If anyone who has an "in" at Skype reads this post, please forward it to your in at Skype.

If any folks are paying for business services from Skype, feel free to let them know you want a Solaris client, and there is an expert on the Solaris audio stack waiting to help.

Thanks!

(On a side note, I'd also like to have VMware on Solaris as well. Yeah, I know about VirtualBox, but I need support VMware for clients, and it would be a heck of a lot easier if I could just host VMware guests on my development head.)

LDRS 29, very cool

2010-06-15T14:56:00.000-07:00

So, this past weekend my son and I went to LDRS 29, which is the event for the national high-powered rocketry club, Tripoli. We were there only one day and one night, but here were some cool highlights from Saturday:

Mass squat launch -- Timothy's Squat with an Aerotech G-67 redline motor flew very nicely, if a bit late off the pad. 28 other rocketeers had their rockets launch at roughly the same time.
Many wild squats. With the $29 specials from WhatsUpHobbies, lots of people were flying very unstable Squat rockets with I-140 skidmark engines. This configuration needs nose weight, as we found out for ourselves when we flew Timothy's with the same engine.
Four half-scale Patriots launched in 3 second intervals from a "box" launch vehicle -- much like a real Patriot. Very, very cool.
Drag race of six or seven N-impulse rockets. These are big rockets, lots of power.
Drag race between a number of very detailed rockets. There was a CATO about 20 feet off the pad, and unfortunately several other rockets were destroyed on ascent by the CATO. Cool to watch, but glad it wasn't one of my rockets.
Full scale O-impulse Patriot launch (and unfortunate catastrophic failure near the end of boost, flaming bits falling down all over the range.)
My own J-350W powered LOC-IV (with significant modifications in nose weight and fiberglass reinforced fins.) This was my Level 2 cert flight and it went brilliantly. (So I'm certified to fly level 2 high powered rockets - impulse up to and including L power.)
Our Big Daddy Estes rocket (typically D or E power) launch with an F-32 Blue Thunder engine -- believe it or not this was one of the highlights for me of the day. As the launch control officer proclaimed -- "a little too much power for the rocket, but we like that!"
Flying a drag race between two D-21 powered 18mm rockets. Both were lost, and later found damaged.
A number of very cool night launches -- lots of creativity here on the part of the rocketeers.
Discover channel. This was a mixed bag... it was cool that they were there, but they did interfere with launch schedules quite a bit. Still, I think we'll be part of the ultimate show, which is supposed to air July 5. Looking forward to watching that.

There was one significant accident, involving an extremely high powered rocket on the far pads. A couple of people were unfortunately badly burned, and had to be medevaced, and our wishes go with them for their recovery.

As for Timothy and I, we're hooked. We'll be going to the November RocStock event as well, provided we can make the schedule work out.

Press release

2010-06-15T14:53:00.000-07:00

Noticed this press release got posted to the Nexenta web site. /me preens. :-)

O_SYNC behavior not honored

2010-06-04T23:14:00.000-07:00

UPDATE (6/21/2010): This problem is apparently solved in b142. Probably other builds as well. But I was unable to reproduce this problem with real hardware on b142.

Note that VMware does not honor cache flushing, so VMware (and possibly other v12n users) will potentially still see this issue.

So, it turns out that ZFS in recent (somewhere after build 134 apparently) builds has a critical bug ... O_SYNC writes are not really synchronous. This leads to potential data loss.

I've not yet figured out which change introduced the bug, but I hope to work on it next week.

In the meantime, I would strongly discourage use of post-134 binaries for anything where data integrity is important.

I've filed a P1 bug with Oracle for this issue. I'll be trying to nail it down further next week; if I'm able to fix it before Oracle can, I'll offer up my fix.

I'll post the CR number when I receive the number back.

I imagine that this bug, which is trivially reproducible, will be getting top priority from the ZFS engineers next week.

UPDATE: CR number is 6958848

The link to access it isn't available yet.

Great Falcon-9 Launch!

2010-06-04T11:56:00.000-07:00

SpaceX, one of our greatest hopes for a commercial manned space program, has achieved a huge milestone with the successful maiden launch of Falcon-9 with a Dragon capsule today. This is the craft that may one day soon be used for ISS resupply, and perhaps even crew transport.

Even as Obama shuts down the US governments manned space program, the commercial sector is picking it up. This is a momentous day.

Congratulations to Elon Musk and the rest of the team at SpaceX!

audioens in VMware...

2010-06-02T22:32:00.001-07:00

So, we have not had audio in OpenSolaris within VMware since... well, ever.

I've been doing some investigation. I'm seeing a situation where the VMware emulated audioens device behaves rather differently from the real hardware.

For one, it seems to insist on using real interrupts. In particular, the sample count registers do not appear to be updated unless one receives and acknowledges an interrupt. (By toggling the interrupt enable bit.) This means that this virtualized device will never be able to run "interrupt free" like the other audio devices (or real audio hardware).

For another, it appears that the audio device has some weird dependency on the relationship between the size of the audio buffer, and the interrupt rate (the number of samples at which to interrupt). Using different values gives, strange results.

Finally, I cannot, for the life of me, figure out how to cause the device to actually trigger an interrupt. I've been able to make some progress by simulating a soft interrupt at 100Hz, which is how the interrupt free framework works anyway, but from what I can tell, nothing is causing a real interrupt to be delivered. This is really strange. (Without this functionality, I am able to process audio at a reasonable rate, but it still stutters, and is not really suitable for real-world use.)

My guess is that the virtual device has some weird dependencies that we don't know about. For example, while the hardware spec identifies registers as being 8, 16, or 32 bits wide, and we use those at the right bit widths, other FOSS drivers all seem to just blithely use 32-bit wide accesses. Is there a hidden dependency here?

If any reader from VMware is seeing this, and can help me understand the behavior of the simulated device, I'd appreciate it. I'd like to make audio work in this environment, if possible. I'm pretty close, I think.

Actually, it seems kind of crazy that these environments emulate such complex audio hardware. (For example sophisticated sample rate conversion hardware.) Much better, I think, would be a simple paravirtualization driver that just exposes a simple buffer and some control functions. If someone at VMware wants to work on that as a solution, I'd be happy to help with Solaris support for it. Since these things run isochronously, and chew up a fair bit of cpu when they run, such a solution would probably be quite useful. (For example, its silly to perform multiple sample rate conversions in software... instead we could express native sample rates via a PV driver to the guest, ensuring only a single SRC operation is performed appropriately in the guest operating system.)

Well That Didn't Work Out So Well

2010-06-01T10:45:00.000-07:00

You may recall my recent blog post about Windows 7 being surprisingly usable.

Well, I have to recant here.

I used Windows 7 for about a week and half. While it *worked*, it was a pleasure to use. But after three BSODs in just that week and half, I have abandoned it. I'm now running Ubuntu. (Why not OpenSolaris? Because I need the ability to host VMware and Skype, and I can't do that natively on OpenSolaris -- yet.)

Sure, I could have called up support -- but Microsoft support is provided by my computer manufacturer, and I didn't feel like spending 3 hours on the phone dealing with tech support while they tried to triage my problem. In the end, it was simply faster and easier for me to reinstall with Linux, even allowing for the time it took to download the media.

Sure, the problem might have been my virtualization software, or maybe it was a shoddy audio driver, or maybe it was brokenness in my graphics driver, or maybe it was the 3rd party antivirus software (which begs the question-- why doesn't Microsoft ship with builtin malware protection -- you'd think given all the heat that they've taken over this that they would have figured out that they *need* a solution here that doesn't involve 3rd parties...)

The "automatic solution finder" that Windows 7 ships was completely unhelpful, it didn't find any links. Google was not much help either... with everything to buggy hardware, drivers, and even overtemp problems being cited as root causes.

I'm sure that tech support would have had me running around in circles trying to solve the IRQL_NOT_LESS_OR_EQUAL blue screen. (I'm guessing, from my kernel expertise, that this is probably an assertion fault somewhere that an IRQ level is set unexpectedly high or low -- exactly the kind of problem I know how to fix in Solaris.) Probably plugging and unplugging hardware, unloading and reinstalling drivers and maybe other software, and generally burning an unmentionable amount of my precious time. Especially given that the hardware tech support I'd have been routed to was unlikely to have any real software clue (which is where I think this problem was most likely located.)

Again, faster and simpler to just dump the busted OS, and load something else.

And, with Linux (or any other FOSS), I have at least a fighting chance of trying to debug the problem myself. Sure, my kernel-fu is substantially higher than average joe home user, so my leanings are more towards something I can troubleshoot myself. But, I will say this, so far I've not had a panic ("oops" in Linux parlance) yet in the past four days that I've been running Ubuntu LTS 10.04 (even though I'm running the "not recommended for general desktop use" 64-bit edition.)

Microsoft, if you see this post, I hope you'll learn something from this.

Last Day

2010-05-21T08:19:00.001-07:00

So, today's my last day as a Sun^WOracle employee.

While I'm excited to be starting at Nexenta, I want to reiterate what I've already said, which is that I've really enjoyed working with the great folks at S'noracle, and that they made this decision to leave quite difficult. Its been quite a ride over the years, and its been fun and exciting. Thanks everyone!

Of course, my old e-mail address(es) at Oracle won't reach me after about noon today.

To reach me for matters pertaining to OpenSolaris, gdamore@opensolaris.org will continue to work. For matters pertaining to my new employer, Nexenta, you can use garrett@nexenta.com. My personal e-mail address of garrett@damore.org remains unchanged. Now please standby while I go reinforce the spam filters...

New Computer

2010-05-21T08:06:00.000-07:00

As part of the process of changing employers, I needed to get a new computer for the new job (and return the old desktop to Oracle.)

I wound up picking this one... I didn't seem to be able to build it any cheaper (as of the date of this post) myself. And guess what... someone goofed! Instead of the 3 GHz Core i7 950, it came with a 3.2 GHz Core i7 960. Bonus! (Other goofs relative to the ad: the system has 9 GB -- but that's spelled out in the details, comes with a black aluminum chassis, and ships with a cheap logitech keyboard.)

I'm still using the stock load of Windows 7, and I'm both surprised (and maybe a bit embarrassed) to admit that the Windows environment (especially when replacing IE with Chrome) is actually quite nice -- fast and usable. Maybe running this environment (and running OpenSolaris in a VM) might not be so bad after all! (Ok, I'll go find some soap to wash my mouth out for blaspheming....) If I do this, besides being able to use Skype for work, I'll be able to use my Phoenix RC flight simulator without having to resort to borrowing the wife's computer...

Engines arrived for Squat yesterday

2010-05-21T07:41:00.001-07:00

The Squat is a 4" diameter short high power rocket with a 54mm engine mount. My engines, 54mm hardware (including the higher end Aeropack retainer), and 38 mm adapter arrived from What's Up Hobbies yesterday. Timothy's going to fly it at the LDRS mass launch on a G67 redline -- this will be his first reloadable engine. Later that day I'll fly it on an I140 skidmark, which represents both my first 54mm engine, and my first Caeseroni engine.

Timothy and I put the rocket together last night; I must say, the higher end metal hardware and thicker fins on this rocket are definitely a step up even from the LOC IV I flew previously on my Level 1 flight (go to about 1:30 in the video link -- I haven't figured out how to edit the video file yet).

I also received the propellant for the J350W, which I'll be flying in my LOC IV as part of my Level 2 certification attempt. OpenRocket says the LOC IV will be approaching 700 mph with this particular engine! Guess I will be glassing the fins on it to help strengthen them for transonic speeds. (I'm open to alternative suggestions from the experts, as well.)

LDRS is going to be fun, indeed!

Greener pastures

2010-05-10T14:05:00.000-07:00

I've recently made a major decision -- I'll be leaving Oracle. My last day as an Oracle employee will be on May 21, 2010. Leaving such a great group of people at Sun will be difficult indeed.

However, I hope to be able to continue as a significant contributor to the OpenSolaris community, as I'll be joining the team at Nexenta. At Nexenta, my responsibility will be to lead a group of engineers working on the OpenSolaris kernel. As such, I'm excited that I'll be able to continue to work on finest operating system kernel on the planet, and I look forward to further collaboration with some of the best software engineers on the planet.

My first day at Nexenta will be on May 24, 2010.

Going up to SF bay area

2010-04-14T14:50:00.000-07:00

Its been a while since I've been to the Bay Area. I'm going up for two days, which is a shade longer than I usually go for. Part of the reason is to make sure I meet with folks in the Bay Area that I otherwise don't see. I'll be up Thursday and Friday April 29 and 30 -- and I expect I'll be at MPK most of that time. Anyone who wants to chat, please let me know.

audiocmihd driver (Asus Xonar cards)

2010-03-17T21:28:00.000-07:00

Some people have been asking me about this driver. (Asus Xonar cards are fairly high-end high definition cards using the CMI 8788 chip.)

I've finally gotten the code reasonably cleaned up, and converted to my interrupt free audio framework.

I'll probably start a case to get this integrated into late b137, or b138. Mostly its just running a bunch of tests at this point.

One problem I have is that I only have Xonar DX1 cards. (PCI.)

If someone is able to help me qualify the driver with build 137 (or a nightly build) of ON, please let me know. The more I can get this driver tested, the sooner I can get it integrated into OpenSolaris.

Interrupt Free Audio

2010-03-16T12:40:00.001-07:00

Today I integrated "interrupt free audio". This set of changes, including some other changes, represents a substantial simplification in the DDI for audio drivers.

The typical audio driver no longer needs to worry about interrupt handlers. On average, about 300 lines of code (or about 10-20% of complexity for typical drivers) was removed from each audio driver.

Furthermore, many audio drivers (for example audio810) are able to run completely lock free, since the audio framework provides synchronization for certain operations. (Operations against each audio engine are synchronized, operations against audio controls are synchronized as a whole, and everything is synchronized against suspend/resume functions.)

Even better, these changes enable some new advanced features that will be used for Sun Ray, virtualization, and hotplug support in the future.

Oh yeah, and since the asynchronous processing now happens as part of the regular timer interrupt, it means that system CPUs can remain in deeper C states for longer, even while playing audio. So, we should have an improvement on system power consumption (admittedly I've not measured this.)

There will be more stuff related to audio in the future, stay tuned.

"Legislative Sleight of Hand"

2010-03-16T12:25:00.001-07:00

I normally have avoided using my blog as a soapbox for my political beliefs. However, I simply cannot remain silent on recent events in the House of Representatives (United States for foreign readers.

No matter what your position is on the health care reforms under consideration, everyone should agree that the reforms are sweeping; perhaps some of the most significant legislation that will affect nearly every American we've seen in quite some time.

House Democratic leadership, knowing that the measure is unpopular with many voters (and hence House Democrats may be unlikely to "vote the party line" to avoid a backlash in their constituencies) are planning a move that is even more offensive than "reconciliation".

While I'm a Republican, and generally opposed to nationalization of 1/6th of our economy, I find far more offensive that the House leadership (particularly Ms. Pelosi) would consider a move that so boldly disenfranchises the people of this nation.

This is a crime, if not against the law, then certainly against the spirit of democracy upon which our country is founded. If health care reform is to be passed, then it should be done with a regular vote where the politicians who vote for it are required to be accountable for those votes (and vice versa, by the way).

If it passes without such a vote, then it will go down as one of the greatest failures of "representative democracy" in history.

Why We Need a Human Spaceflight

2010-03-15T12:20:00.000-07:00

Space aficionados may be aware that President Obama has canceled the previous administration's "Vision for Space Exploration", which consisted of the Constellation program including Ares I, Ares V, and Orion. This has been fairly well covered in the mainstream media.

Critics of the Constellation program raise some significant and relevant objections to the Constellation program.

However, I strongly believe that as a nation, we need a national space program that includes human spaceflight beyond low Earth orbit. The cancellation of Constellation, while perhaps with good cause, has left our national space program with a vacuum -- the lack of a heavy lift vehicle, and lack of any vision, would effectively constrain human exploration to LEO for a generation. Furthermore, it significantly constrains the kinds of activities that we can perform in LEO.

Its my belief that this is short-sighted in the extreme.

We need a space program that includes vehicles with the ability to loft large payloads into orbit. Projects like the International Space Station, and further commercialization of space, are only possible with the ability to loft a significant payload into orbit.

We also need to plan for human exploration beyond our front porch. While many people argue that sending robotic explorers is less risky, and far cheaper, the idea that we can or should abdicate all future space endeavors to robotic missions is actually offensive.

Robots won't inspire a generation of students to continue to excel at math and science. Robots can't stand in as national heroes. And robots alone won't help develop the enthusiasm required for the general public to continue to want to invest in space and space technologies. Robotic exploration is mostly a solved problem -- many new technologies that are necessary for human space travel will simply not be invented or invested in, without the "problems" to solve that are involved in human space exploration.

I'm from a generation of kids who viewed astronauts as near personal heroes; I dreamed, and still dream, of being able to see our planet from space itself one day. I dream of the days when human kind steps beyond just Earth, and has outposts on the Moon, Mars, the asteroids, and perhaps other interesting places in the solar system. And someday beyond.

My own son dreams someday of being an astronaut and visiting Mars. Unfunded as it was, at least VSE allowed a glimmer of such a hope. Obama has killed that hope, and maybe the dreams and hopes of thousands or millions of other like minded kids.

Fortunately, there is a proposal that would revive these dreams, and allow us to retain a national heavy lift capability, retain a lot of the knowledge and expertise that we acquired with the successful STS (space shuttle) program (even reusing a significant amount of the materials and technology), and allow for a "way forward" that would allow us to get beyond LEO and go to interesting places elsewhere in the solar system. The DIRECT v3.0 proposal is IMO the best way forward; it allows us to have our cake and eat it too -- giving us all the heavy lift capabilities that we need, minimizing the significant impact on our economy that both the Constellation program, and the cancellation of the STS and Constellation programs, create.

I firmly believe that we are on the cusp of a major economic shift, where commercialization of space may play as important a role in the coming decade or two as the Internet has played in the previous two. The question is, will we as a nation continue to develop that potential, or will we let it slip away, to be picked up by India, China, or Russia?

Yes, I'm an American. And I believe that it is important for America to be a leader in the exploration and utilization of space. Ultimately, I believe that "planting flags" is much more important than the proponents of solely robotic exploration would have us believe. Someday people will visit Mars. Will America be there, or will we just be an observer while one of the Asian nations celebrates a major achievement?

ON IPS surprisingly easy

2010-03-03T19:09:00.000-08:00

So I have an EOF RTI that was in queue when the ON IPS integration happened last night.

Of course, this totally whacked my packaging changes, and I had to modify them. Making the changes was quite easy. Here's the old, and the new version of the changes. Its actually less files to update under IPS.

I was dreading retesting. Dealing with distro construction sounded "painful".

I needn't have worried. In the tools directory there is this neat tool called "onu" (on-update I guess?)

I had to load a machine with b133 to set up a baseline, but we have a nice way to do that internally via our internal infrastructure and AI. It boils down to running one command on an install server than doing "boot net:dhcp - install" at the OBP prompt. (Yes, this is a SPARC system.)

It took a little bit for it to install, but less than an hour.

Then, after rebooting and getting the initial settings on the system, it was just a simple matter of "onu -d ${ws}/packages/sparc/nightly-nd" to update it. This took a while (20-30 minutes, I wasn't counting). Eventually the system was up and ready for business. Easier than bfu. Amazing.

Thanks to the IPS team! I can't wait for bfu to finally go away.

Funny Ancient Software

2010-02-21T21:35:00.000-08:00

I just found out that Ubuntu has been shipping (since version 6.06 -- Dapper Drake I think it was called?) and apparently included all the way into forthcoming 10.x LTS version) a program I wrote nearly two decades ago as a student -- vtprint -- and yes, that link points to manual text I wrote way back when.

(This program, "vtprint", was for use with printing from a UNIX shell prompt, when you don't have a better way to move files around. Back then we used commands like "kermit" to connect to a UNIX server from our PC over a 2400 or 9600 baud modem -- and well before PPP or even SLIP.)

I haven't used vtprint since about 1995, but its funny to still see it kicking around. Too bad the docs still have an old e-mail address for me at SDSU.... I guess nobody has needed a bug fix for it for some time.

Congratulations to BMW Oracle Racing

2010-02-15T12:28:00.001-08:00

If you're involved in the sailing community, you'll already know that Larry Ellison, who's now ultimately my boss, had put together a team to challenge the America's Cup. They won this weekend, bringing the America's Cup back home to America, and I'm enormously proud of Ellison and his team, both as an American, as a sailor, and -- now -- as an Oracle employee.

Open Development

2010-02-12T11:59:00.000-08:00

Note: I'm posting this on my personal blog, which as always is a reflection of my own thoughts and in no way represent any official policy from my employer (whoever that may be).

Now that we former Sun employees (for the most part) are now part of a larger company, there have been some questions about how much of the trend Sun had made towards open development will continue (particularly where Solaris/OpenSolaris is concerned.)

(I want to separate the concern of Open Source -- where source code is made available for products after they are released -- from Open Development -- where the product is developed in the open.)

Many of us who were part of that acquisition are wondering the same things. Officially, the word is "no changes in what we're doing", but unofficially there's an atmosphere that our new employer places a greater emphasis on commercial profitability and a lesser emphasis on things like "including the community."

Speaking abstractly, there are risks to any open development effort, particularly when the effort is intended to be supportive of a commercial endeavor. The risks range from enabling competitors with early information, to forestalling customer purchases of bits today as customers wait for the new feature that's being developed in the open, to simply diluting the impact that "surprise" delivery of a new product or feature can make.

Certainly there seems to be some evidence that Oracle may have a greater concern about the costs and risks associated with "early disclosure" than Sun did. No matter how passionately one may believe in Open Development, nobody can deny that there are real costs and risks associated with open development.

So the challenge for Oracle is to figure out what the balance is that makes commercial sense.

Ultimately profit is the primary responsibility of any publicly traded company.

The challenge for the community is to figure how to provide commercial justification to Oracle for the open development that the community likes to see.

If you want to retain truly Open Development (which includes public posting of webrevs for stuff developed internally, open ARC reviews, and public design discussions on mailing lists and similar fora) of Solaris and OpenSolaris, this is call out to you.

Have you or your company made a significant Sun purchase? Has open development (as opposed to open source) influenced your decision? How and why? Will open development influence future purchasing decisions? If you can, put a number to the value of open development, and provide that information to your sales reps or post it publicly.

The decision makers need to see value in the practice of open development if they're going to continue to support it.

Again, I'm only talking about open development, not about open source.

As an aside, I don't think statements coming from community contributors without the support of purchasing dollars are likely to carry much weight with Oracle decision makers. I believe that if you look at the contributions from the community-at-large in OpenSolaris, you'll find that the meaningful contributions have been fairly small and generally of little commercial interest, and have always required additional engineering expense from Sun. So "leveraging" the community for development has not, IMO, been a gamble that has yielded dividends -- at least not from the perspective of either Sun or Oracle.

Scalability FUD

2010-02-04T09:41:00.000-08:00

Yesterday I saw yet another argument about the Linux vs. Solaris scalability debate. The Linux fans were loudly proclaiming that the claim of Solaris' superior scalability is FUD in the presence of evidence like the Cray XT class of systems which utilize thousands of processors in a system, running Linux.

The problem with comparing (or even considering!) the systems in the Top500 supercomputers when talking about "scalability" is simply that those systems are irrelevant for the typical "scalability" debate -- at least as it pertains to operating system kernels.

Irrelevant?! Yes. Irrelevant. Let me explain.

First, one must consider the typical environment and problems that are dealt with in the HPC arena. In HPC (High Performance Computing), scientific problems are considered that are usually fully compute bound. That is to say, they spend a huge majority of their time in "user" and only a minuscule tiny amount of time in "sys". I'd expect to find very, very few calls to inter-thread synchronization (like mutex locking) in such applications.

Second, these systems are used by users who are willing, expect, and often need, to write custom software to deal with highly parallel architectures. The software deployed into these environments is tuned for use in situations where the synchronization cost between processors is expected to be "relatively" high. Granted the architectures still attempt to minimize such costs, using very highly optimized message passing busses and the like.

Third many of these systems (most? all?) are based on systems that don't actually run a single system image. There is not a single universal addressable memory space visible to all processors -- at least not without high NUMA costs requiring special programming to deliver good performance, and frequently not at all. In many ways, these systems can be considered "clusters" of compute nodes around a highly optimized network. Certainly, programming systems like the XT5 is likely to be similar in many respects to programming software for clusters using more traditional network interconnects. An extreme example of this kind of software is SETI@home, where the interconnect (the global Internet) can be extremely slow compared to compute power.

So why does any of this matter?

It matters because most traditional software is designed without NUMA-specific optimizations, or even cluster-specific optimizations. More traditional software used in commercial applications like databases, web servers, business logic systems, or even servers for MMORPGs spend a much larger percentage of their time in the kernel, either performing some fashion of I/O or inter-thread communication (including synchronization like mutex locks and such.)

Consider a massive non-clustered database. (Note that these days many databases are designed for clustered operation.) In this situation, there will be some kind of central coordinator for locking and table access, and such, plus a vast number of I/O operations to storage, and a vast number of hits against common memory. These kinds of systems spend a lot more time doing work in the operating system kernel. This situation is going to exercise the kernel a lot more fully, and give a much truer picture of "kernel scalability" -- at least as the arguments are made by the folks arguing for or against Solaris or Linux superiority.

Solaris aficionados claim it is more scalable in handling workloads of this nature -- that a single SMP system image supporting traditional programming approaches (e.g. a single monolithic process made up of many threads for example) will experience better scalability on a Solaris system than on a Linux system.

I've not measured it, so I can't say for sure. But having been in both kernels (and many others), I can say that the visual evidence from reading the code is that Solaris seems like it ought to scale better in this respect than any of the other commonly available free operating systems. If you don't believe me, measure it -- and post your results online. It would be wonderful to have some quantitative data here.

Linux supporters, please, please stop pointing at the Top500 as evidence for Linux claims of superior scalability though. If there are some more traditional commercial kinds of single-system deployments that can support your claim, then lets hear about them!

Missing audio packages

2010-02-04T00:55:00.001-08:00

I have learned that at least two packages, SUNWaudioemu10k and SUNWaudiosolo, are not part of the "standard" ("entire?") install of OpenSolaris b131. If you're looking for either of these, you should do "pfexec pkg install SUNWaudiosolo" or "pfexec pkg install SUNWaudiosolo".

Hopefully we'll get this sorted out before the next official release.

Update: Apparently (according to the expert I talked to) this problem only affects systems updating with pkg image-update. If you install a fresh system, the audio packages should be installed.

System board for ZFS NAS

2010-02-02T12:28:00.000-08:00

I'm thinking about creating a home storage server, like many, and I want it to be performant enough to host work spaces for compilation over NFS, and efficient enough to reduce my current power consumption somewhat.

I'm thinking of a new Intel D510 system board, and looking at several, I found a board from Supermicro that looks ideally suited to the task. Does anyone else have experience with this board? It looks like its all stock Intel parts, so it should Just Work.

I'm thinking that with 4 or more SATA drives combined with RAIDZ, and dual Intel 82574 gigabit Ethernet (which I could use in an Ethernet link aggregation), I should be able to get excellent performance. (I might even set up jumbo frames, to further bump NFS performance -- if they really are 82574's then they support up to 9K MTU).

Kindle Converts a Skeptic

2010-02-02T12:06:00.000-08:00

Recently I bought my wife an Amazon Kindle (the new international unit), at her request. Personally I was rather skeptical -- trying to read book material on computers, even laptops or netbooks, has always felt very awkward to me. I always believed that there was something about holding a paperback (or even a hardback) which would never be replaceable by technology -- maybe for others, but at least not for me.

I have to recant. Debbie has read something like a dozen novels already on her unit. I decided to try it out... and I have to say, I was surprised. I was reading H.G. Wells' War of the Worlds (not for the first time of course), which was a free download, and wow, was I surprised. After 10 or 15 minutes of reading, I almost forgot I was holding something in my hand that isn't printed paper. (The form-factor, which is quite similar to a book, works quite well here. I don't think I'd like the larger DX, as it would destroy the "illusion" of reading a paper back book.)

Not only did the technology not "get in the way", the reading experience was actually more pleasurable, largely because I was able to bump up the font size up to a more comfortable reading level. Last night I read about 1/3 of the book, before I got too tired, but I'm sold on the concept -- and I was a die hard skeptic before.

I don't think I'd like to use it for other things ... but for the primary purpose of reading novels, it settles in quite nicely bringing some technological advantages without letting the technology get in the way of reading.

Will Apple's newer iPad compete here? I'm skeptical. The Apple product is a fancier device, with its backlit screen, and probably will feel more like a hybrid between a laptop and an iPhone (of course I still have an ancient model of phone that is used pretty exclusively for making phone calls -- call me a Luddite.) I suspect that the combination of screen glare, snazz, and lower battery life (iPad users will need to be lot more cognizant of their current battery status), means that its going to be a poor replacement for a Kindle, and an even poorer replacement for the printed materials that the Kindle is meant to replace.

When I go on my round-the-world sailing trip (not any time soon!), would I want a Kindle with me? Absolutely (or something similar) -- along with a solar or wind based charging system. (Product idea... a case for the kindle that integrates photovoltaic solar charging system, so your Kindle is always charging when its closed.)

An iPad? Not likely -- if I'm going to be working or sending e-mails, sure, but then one of the netbooks is probably a better option.. with a "real" keyboard.

Reprehensible behavior from a monopoly

2010-02-01T09:55:00.000-08:00

Misbehavior stemming from lack of competition is apparently not unique to the IT industry.

I saw this post today, and couldn't believe it. And then a bit of additional research shows this is not unique -- a number of people complained about actions on the part of Greyhound that would never be tolerated in market where there is true competition.

Forcing a grandmother to wait out in cold, while there's still snow on the ground, may not be in violation of the letter of the law, but it is certainly in violation of the basic tenets of human decency, and the management at the Memphis location showed they have none.

Its been over ten years since I've ridden a Greyhound (or any other long-haul bus for that matter), and after reading this, I am unlikely to ride another Greyhound again. Instead I'll stick to air transport where lively competition means that even the worst airlines understand that they have to at least pretend to care about their customers.

If you're reading this and thinking about taking a Greyhound somewhere, don't.

While there may not be much competition for Greyhound for long-haul ground travel, there is at least some. And there is always air transport for those able to use it.

I'll be interested to hear if Greyhound corporate does anything to fix the problems they obviously have. A good start would be firing most or all of the staff at their Memphis location (especially the management and security guard in question) and refunding the tickets of each of the passengers who were stuck there.

Its Official

2010-01-27T07:39:00.001-08:00

I'm no longer a Sun Microsystems Employee, since Sun no longer exists. Hopefully I'll get to keep my job at Oracle, but I've not seen any e-mail yet. I expect I will before day's end.

Auto Install Finally Working For Me!

2010-01-25T22:51:00.000-08:00

Some of you may know, I've been struggling (and failing) to make auto install work for me. I've had challenges, because my network is not routable, and due to other issues (bugs!) in OpenSolaris Auto Install.

However, it seems that I've finally hit on a successful recipe. I want to record this here for others.

First, in order to use AI, you will need your installation server to be running a recent build of OpenSolaris. The release notes indicate b128 is sufficient. I just ran "pkg image-update" to update to build 131. If you fail to do this, there won't be a warning at all, its just your clients will simply not boot.

The next thing you'll need to do is download a full-repository and the AI image.

Unfortunately, there are not public versions of the full repo ISO file available that are "current". (No, I can't get you a copy, and no I don't know why they haven't posted a more recent update.) Hopefully this problem will be corrected soon.

Setting up the local repository can be done following these directions. (Note that you will have to change the paths to reflect your system. I stash ISO images in /data/isos, and install images under /data/install. These are separate ZFS filesystems.

# where do ISOs live, without leading /
ISODIR=data/isos
# where does the repo live, without leading /
REPO=data/install/os131_repo_full
# AI service name to use
NAME=os131_x86
# parent directory for installations, without leading /
INSTDIR=data/install
# port to use for install server
PORT=8181

mount -F hsfs -r /${ISODIR}/osol-repo-131-full.iso /mnt
zfs create -o compression=on ${REPO}
svccfg -s application/pkg/server setprop pkg/inst_root=/${REPO}
svccfg -s application/pkg/server setprop pkg/readonly=true
svccfg -s application/pkg/server setprop pkg/port=${PORT}

You'll need to edit the ${REPO}/cfg_cache file, changing the origins entry to match your system. I used a value like this:

origins = http://192.168.128.11:8181

Then you'll want to use installadm to setup an initial boot service. Here's the recipe I used:

zfs create -o compression=on ${INSTDIR}/${NAME}
installadm create-service -n ${NAME} -s /${ISODIR}/osol-dev-131-ai-x86.iso /${INSTDIR}/${NAME}

Now you need to change the default manifest file. This is the tricky part, that IMO was not terribly well explained anywhere else.

cp /${NSTDIR}/${NAME}/auto_install/default.xml /tmp

Then edit default.xml file in /tmp, changing the value of "main_url" to point to your server. I used a value like this:

<main url="http://192.168.128.11:8181" publisher="opensolaris.org"/>

Then apply this manifest to the default manifest:

installadm add -m /tmp/default.xml -n ${NAME}

Finally, I did some tweaking in my DHCP configuration. I have a macro for each service name, that provides the defaults. For example, my "os131_x86" macro looks likes this:

Include pepper
BootFile os131_x86
GrubMenu menu.lst.os131_x86

My "pepper" macro (pepper is the name of my server) sets some shared defaults, but most especially it sets BootSrvA to the IP address of the server (192.168.128.11 in my case.)

Then I just configure individual addresses for which ever version of OpenSolaris (or SXCE) I want to install using the the correct configuration macro. (For SXCE there are very different DHCP options to use. Also the SPARC version of OpenSolaris uses different options as well.)

Six Years & Counting

2010-01-19T07:34:00.000-08:00

Its hard for me to believe that six years ago today at this hour in the morning I was getting myself ready to meet my bride. We had a wonderful wedding on the beach in front of the Del Mar powerhouse in San Diego, with our friends and family in attendance.

Looking back, its been the best six years of my life. I've truly been blessed. I'm looking forward to spending the next sixty together with my beautiful bride Deborah.

Interesting device driver work

2010-01-14T08:26:00.000-08:00

So there are a couple of "closed" drivers that are not part of OpenSolaris, and might never be because of redistribution restrictions. However, this represents an opportunity for an enterprising software engineer to contribute. The drivers are

glm - Symbios 53x810 and similiar devices
qus - QLogic ISP 10160 and similar devices
adp - Adaptec AIC 7870P and similar devices
cadp - Adaptec AIC 7896 and similar devices

There are open source drivers for these from FreeBSD and NetBSD, which could be used as a starting point for a port. I'd probably be interested in trying one of these out myself, if time allowed -- but alas it does not, my plate is already quite full.

The best part of these drivers is that there are few, if any, "political" or "business" restrictions on integrating replacement drivers. Indeed, at one point recently each of these was considered for an EOF simply because they weren't considered strategic anymore. (The EOFs were rejected, but these will only be delivered via an extras repository or somesuch.)

So, what are you waiting for? This is a good opportunity to learn about SCSA, and provide us with superior replacement drivers. (The glm replacement looks like it could be done in as few as 2 or 3 KLOC; that is all the NetBSD version of the driver uses.)

A "modern" elxl driver

2010-01-13T02:49:00.001-08:00

I undertook this past weekend an effort to "port" the NetBSD "ex" driver to GLDv3, as an open source replacement for "elxl".

It took a bit over 2 days.

The new sources are on line in a webrev format. I'd really appreciate feedback. I'm hoping to integrate these changes soon (and I could use help with testing!)

This new version, apart from being Open Source, also has with it:

1. Full support for VLANs (including full-MTU frames)
2. Full support for link notification on twisted pair (and hopefully also fiber)
3. Full integration with Brussels for MII and media selection.
4. Full support for Suspend/Resume (S3)
5. Full support for Quiesce (fast reboot)
6. Support for additional devices

It can also be extended fairly easily to support Cardbus and MiniPCI variants, and hardware checksum offload. (The checksum offload part is basically written and #ifdef'd out until I find a newer card to test with personally.)

What is missing is "automatic" media selection based on active probing. The old Solaris driver would "autoselect" which port (BNC, AUI, twisted pair) to use based on some active probes to look for link. These were rather complex, and not something I could take from the closed driver. These days everyone just uses twisted pair anyway, right? (The fiber and TX4 cards don't offer any choices, so there is no probing needed for them.)

If you have such a COMBO card, you can force the media using a new "driver private" property called "_media", which you can set to various values. See the driver sources in the webrev for more information.

I've done enough work on this driver that there is probably as much of my own code in it (at least) as was in the original NetBSD code. Nonetheless, I'd like to than the NetBSD Foundation for making these sources available.

I'll be posting binaries soon, stay tune. (Sun internal users can grab the binaries from /net/temecula.sfbay/data/work/gdamore/yge/yge/usr/src/uts/intel/elxl/ -- at least for now. That path is likely to work until I get the code integrated.)

Out with the Old, In with the New

2010-01-01T12:44:00.000-08:00

Happy New Year (2010) everyone!

I thought I'd take a second to reflect on the accomplishments of the past year, and look forward to what I think is in store for my contributions to OpenSolaris this year.

It's hard to believe that I've been a member of the OpenSolaris community for over 5 years now. (I was a pilot member.)

Undoubtedly this past year my biggest contribution to OpenSolaris was the new audio framework (Boomer) and many new audio drivers.

I did a lot of other work besides, including a bunch of work on NIC drivers (including a new common MII framework and the yge driver which supports Marvell Yukon 2 parts), and various changes to the SDcard framework.

I've also developed a device driver for a very interesting (and very high performance) hybrid storage device (which won't be integrating for non-technical reasons), and a new storage framework for block oriented storage devices. (This framework, blkdev, will be integrating once we're out past the build restrictions.)

I also did a lot of cleanup of legacy and stale code, which hopefully reduces the install foot-print and shrinks compile times somewhat.

2009 was also the year I became a full voting member of PSARC, and I was privileged to serve 3 months as PSARC chair. (The chair rotates amongst all active PSARC members on a roughly quarterly schedule.)

This past year is also the year that I became the top contributor to ON in separate integrations, since the OpenSolaris project started, at least as reported by ohloh.net. (Note that the statistics only cover the open portion of the ON consolidation.)

So what's coming up in the next year? Here's what I expect to be working on:

10GbE ethernet for Mellanox ConnectX devices (hermon). This is probably my top priority at the moment. The work is largely being done by Mellanox, but I'm the Sun engineer ultimately responsible.
Sun Ray audio. This is one of my biggest priorities. We want a Boomer driver for Sun Ray audio, bringing in-kernel mixing to Sun Ray appliances.
Interrupt-less audio. This is a major rethink of the way we process audio in the kernel, and reduces a lot of complexity in device drivers and is an enabler for several other new features. The code for this is done, so expect an integration in b135 or 136.
Better audio hotplug support. (Especially for USB audio.)
Better audio virtualization support (especially with Trusted Extensions.)
Additional audio device support. (Asus Xonar, via audiocmihd, for example.)
Integration of blkdev, and hopefully faster, simpler, better kernel support for simple block oriented flash-storage devices. (I.e. those devices that don't natively understand the SCSI command set.)
Fixes for a variety of network device driver bugs. I've already got a few of these changes queued up.
Broader support for the MII framework in other NIC drivers. (I have changes queued up for rtls, already, for example.)
Further cleanup of not-needed legacy code.
Continued contributions and participation at PSARC.
Support for SDXC card media (and possibly also development of exFAT filesystem code, dependent on licensing concerns.)
Possibly work on various track pad bits, depending on time and resource. (Synaptics support finally?)

There are probably other things that will happen, and I'm sure that our new owners at Oracle (assuming that the deal gets finalized, which looks almost unstoppable at this point) will have some ideas about how my priorities will be spent.

One thing is that I feel very privileged to have been able to work on the OpenSolaris code base, and to continue to be able to do so. I often reflect that its amazing that I get paid to do this work -- I think I'd be far less productive if I didn't enjoy my job so much. When my management asks me to take a break and do something fun for a change, I think he has a hard time understanding that for me hacking on OpenSolaris code is fun. I genuinely hope that I continue to be so privileged for the foreseeable future.

Shortest Audio Driver?

2009-12-26T19:34:00.001-08:00

Using my new framework changes (which have not integrated yet), along with changes in the framework designed to simplify drivers by providing certain ordering guarantees, I've been able to shrink a lot of the audio drivers (and in the process have also found a number of small bugs that I think ultimately were responsible for jitter and corruption). Ths simplest driver so far -- audiovia97, which is about as simple an AC'97 driver that can be, is only 686 lines. Several others are now less than 800 lines, and no longer need any locking primitives. As far as device drivers go, this is minuscule. In fact, I challenge anyone to identify a driver that fully services any "real" piece of hardware, fully supporting said hardware, in fewer lines of space.

(And in case you are wondering, a significant fraction of those 686 lines of audiovia97 is still comments and whitespace and even the 30+ lines of boilerplate -- the driver is written to be as readable as reasonably possible, at least to other device driver or kernel engineers.)

The forthcoming audio DDI will be about the easiest DDI for any kind of device driver in Solaris. Contrast that with the pre-Boomer SADA framework, which one could argue had the most cumbersome and painful DDI for non-nexus devices.

Hopefully the simpler framework will make it easier for others to provide new drivers, and result in far fewer "bugs" in the drivers. (And of course, the smaller drivers will result in more efficient use of kernel memory as well, although you are unlikely to notice that particular difference.)

(However, the complexity of the audiohd driver remains a serious thorn in our side -- and unfortunately the problem is made worse by a far-too-complex specification for the Intel HD audio system That said, the complexity in audiohd stems largely from offering too many codec configuration choices, and not from the part of the driver responsible for actually transporting audio data.)

Do you use IDN? (E10K inter-domain networking)

2009-12-22T21:39:00.001-08:00

I was looking, and one of the legacy device drivers we still support is the Enterprise 10000 Inter-Domain Network. This is a memory based interconnect network that emulates an ethernet.

It occurs to me that we could update this driver to use the GLDv3, which would potentially have a significant performance boost for it. (Really, a better solution would be to create a new low level MAC type for it Nemo, but clearly that's probably too much pain for a product that is no longer shipping).

Anyway, I'd be interested to hear of any sites which use IDN on E10K class systems, and would like to see this (already high performance) network get another boost.

The other legacy SPARC drivers which are not GLDv3 compliant are:

cassini (ce) (probably never going to be GLDv3, but that's another matter)
gem (ge) -- this one could be easily converted *if* I had fiber cards to test it with, although there are political concerns as well. (It uses the same MAC controller as "eri", which I converted a while back.)
scman -- This is used for the Starcat (E15K/25K) connection between the SC and domains. It is also based on the eri driver, but has some special hacks. I'm a bit paranoid about touching this one, but if someone really wants it to be GLDv3, please let me know. (This isn't supposed to be a high performance network!)

I'm not aware of any other non-GLDv3 device drivers remaining for SPARC, but if you know of any, please let me know!

New audioctl coming soon

2009-12-16T22:08:00.001-08:00

I integrated a replacement for mixerctl (and also removed mixerctl) back in build 130. Well, build 130 is coming out really soon (I believe it is available internally in SXCE form now.)

I think therefore that it is time to provide more detail here about the new audioctl syntax:

NAME
   audioctl -    audio mixer control command line application

SYNOPSIS

   audioctl list-devices

   audioctl show-device [-v] [-d device ]

   audioctl show-control [-v] [-d device] [control ...]

   audioctl set-control [-v] [-d device] control value

   audioctl save-controls [-d device] [-f] file

   audioctl load-controls [-d device] file

DESCRIPTION
   The audioctl command is used to control various features  of
   the audio mixer and to get    information about the audio mixer
   and the audio device.

   The audioctl command operates on the following data types:

  device
      An audio device, such as "audiohd#0".   The subcommands
      that accept this do so as an argument to an option -d.
      If not supplied, then the default audio device is assumed.
      Any device node associated with an audio device will work
      as well, such as /dev/sound/0, /dev/dsp1, or /dev/audio.

  control
      A mixer control name, such as "volume".

  value
      The value of a control.  The specific format depends on
      the type of control.  Monophonic values usually use a single
      whole number between 0 and 100, inclusive.  Stereo values
      use a pair of such numbers (representing right and left
      channels.)  Boolean values indicate either "on" or "off".
      Enumerations take a single value of one or more names.

  file
      An ASCII text file of control settings.

Options:
   Each subcommand has its own set of options that it takes.
   However, some subcommands support the special flag -v, which
   indicates a request for more verbose output.

SUBCOMMANDS
   The following subcommands are supported:

   audioctl list-devices

  List all the audio devices on the system.

   audioctl show-device [-v] [-d device]

  Display general information about a device.

   audioctl show-control [-v] [-d device] [control ...]

  Display the control setting values for the device.  The named
  controls are displayed.  If no control names are provided, then
  all control values are displayed.

   audioctl set-control [-v] [-d device] control value

  Changes the value of a control to the supplied value.

   audioctl save-controls [-f] [-d device] file

  Saves the current state of all mixer control values to the named
  file.  The command will abort safely if the file already exists,
  unless -f is supplied.

   audioctl load-controls [-d device] file

  Restores previously saved state in the named file for all mixer
  controls.


ENVIRONMENT VARIABLES
   AUDIODEV     If the    -d and -a options are not specified,  the
       AUDIODEV  environment    variable is consulted. If
       set, AUDIODEV contains    the full path name of the
       user's     default  audio    device.

I expect there may be some questions or concerns about the new syntax. If they are not answered by the manual page above, please don't hesitate to ask me.

quick system identification

2009-12-14T11:06:00.000-08:00

I am frequently needing to know "which system am I using". (I have trouble keeping track of which ones are which) -- i.e. is this the laptop, the Ultra 20, the whitebox, etc.

I had been using smbios a lot, but that's a bit awkward and non-portable to SPARC or systems that don't support smbios. Here's a quick script that can quickly tell me what model I'm working on. It works on SPARC and x86 alike. I have it stored in ~/bin/system -- it only works on Solaris of course. Feel free to borrow, reuse, whatever -- I hereby donate it to the public domain:

#!/usr/bin/ksh

sysconf=`/usr/sbin/prtdiag | head -1`
biosconf=`/usr/sbin/prtdiag | head -2 | tail -1`

sysconf=${sysconf#*:}
biosconf=${biosconf#*:}
sysconf=`echo ${sysconf} | /usr/bin/sed -e 's/^ //g'`
biosconf=`echo ${biosconf} | /usr/bin/sed -e 's/^ //g'`
if [ -z "$sysconf" ]
then
printf "BIOS: %s\n" "$biosconf"
else
printf "%s\n" "$sysconf"
fi

Sample outputs from a Toshiba laptop, a Sun V890, and a non-name Intel whitebox system:

TOSHIBA TECRA M9
Sun Microsystems sun4u Sun Fire V890
BIOS: Intel Corp. BX97520J.86A.2777.2007.0805.1747 08/05/2007

While the format isn't terribly parseable, its great for giving a quick assessment about what type of machine I'm on.