Up until very recently I had never heard of Dtrace.
Sun Microsystems, makers of brilliant software but not money, created Dtrace for their Solaris OS. It allows you to create probes and listen on a port where syscalls or other OS events are reported.
This is awesome and also ported over to Mac OS X. This set of one-liners gives a small example of what's possible. If you're on a mac or FreeBSD or Solaris (!) and looking for a good time try running a few of these.
And if you have Instruments installed, you can launch apps and monitor their calls, file I/O, memory leakage, etc...
For my own personal Rube Goldberg machine, I might write a Dtrace script which listens for whenever files of a certain type are opened, and pipe them to a script which automatically backs them up using some version control.
Or similarly, automating other tasks such as running make whenever code files are written.
I was excited to learn about and had never heard of Dtrace, so I thought I'd share, enjoy!
Friday, July 10, 2009
Thursday, June 4, 2009
11 week Project Manager Manifesto
If you are taking a project course that lasts 11 weeks, and you are leading a group of 5-8 software engineers, congratulations, you are a project manager for CSE 403!
Read this post-mortem fresh from the source, the project manager on Robot Rock. Robot Rock is an open-source interactive music framework built in 11 weeks by 5 UW undergrads for CSE 403, a software engineering course.
Suggested reading:
The Pragmatic Programmer
How To Win Friends and Influence People
You are the one to set the tone. Work as hard as it takes to get the project off the ground, and demonstrate what your expectations are. The sooner you demonstrate that the project could be a success, the sooner you win the rest of the team's dedicated efforts, and you can't get far without that.
This is the time for experimentation--it will never get easier to test out different concepts!
For example, Robot Rock started with separate demos of drum sounds in a loop and a trivial non-functional UI. Mashing these together was a major milestone and morale boost, demonstrating the fact that all of our technologies can live and work together in the same environment in a doubtful time when we still struggled to set it all up.
It also provided a tangible way to demonstrate that a lot of the hard work was already done, and was useful for settling disputes through experimentation instead of abstract arguments (for example, when designing architecture based on the performance limitations of our libraries).
Establish regular meeting times during the week, and hold onto this inertia. Meeting often is the best thing your group can do.
Don't think of yourself as any sort of authority, think of yourself as the sniffable glue keeping your team happy and productive and it's up to you to figure out exactly how to make that happen! If you read Dune you know that as soon as you start giving orders, people stop acting autonomously, so don’t command, lead instead.
Write lots of tests in the beginning, but keep them fairly general/flexible since things will be fairly tumultuous at the start and tests will die quick and noisy deaths.
You need the tests for just two purposes: to verify and enlighten. Tests help maintain confidence and momentum, and allow you to breath easy knowing that you aren't breaking anything as you hurtle with breakneck speed through the rapid development which takes place after the design is settled. Of course, with all this code, you need a way to demonstrate what it all does. Tests are a great way to communicate to your team mates how to actually use the code you're writing.
Don't let tests prevent refactoring--accept that you will have to some tedious effort fixing them up. If you hate the thought of this, don't go hog wild writing tests for things likely to change, and write professional tests which don't have lots of repeated code.
Don't leave your tests broken.
Assembla is a hugely valuable tool. It integrates a ticket system, file hosting, milestone tracking, wiki, and repository with online code browsing.
Use the wiki to maintain your documentation and deliverables. Use the milestone tracking to organize and shuffle your tickets around in time.
As soon as you think of something that needs to be done, create a ticket for it.
So, you're in 403, which means you're writing code for 11 weeks, then throwing it away, right? There's some truth to this, but if you write ugly code you can't use it to convince people to hire you. Often job postings require an example of code you've written to solve a non-trivial problem, and so this is your chance to shine to all future employers.
There are a lot of general things which constitute a good design, such as modularity, simplicity, modules which are decoupled, interfaces, etc.
This isn't really about bad design/good design though. You want a design which will make your high level goals natural to achieve.
For example, we knew at the start that it was critical for Robot Rock's music to feel very responsive and sensitive to user manipulations.
We also knew that we wanted the audio generation to have no influence over the rest of the design, in case we wanted to drastically change the way we generate the actual musical tones.
From these first principles, we could evaluate each potential design (everyone came up with a rough design as a homework assignment) and examine if it allowed for real time responsiveness, as well as decoupled abstract song data from the underlying generation. Once these requirements were met, we went on to examine other more general qualitative characteristics, such as separation of components and simplicity.
If we started with an arbitrary design goal, such as simplicity, we may have locked ourselves into a design where real time responsiveness was anything but natural, despite whatever other elegance is achieved. In other words, we would have picked A Good Design, but it'd be The Wrong Design.
Build the simplest thing that meets the requirements.
If there's a feature you want to add, consult your users first. Make a prototype before investing the time in something which the user may not actually care about.
No optimization until feature complete, or later even.
Understand when code is good enough...I'm just parroting The Pragmatic Programmer at this point, so just read that instead.
Assign each group member an area of ownership. It doesn't have to be something they are familiar with, just something they care the most about ideally. There should be no part of the project without an associated leader. Everyone can just refer to this person for questions, and it will deepen their understanding of the respective area. It also provides resume bullet points.
Not everyone will have the same knowledge level of the languages you use. It helps to have at least one person experienced with each language utilized, and they should oversee code reviews with the less experienced members, at least during the initial stages. It goes without saying that the instruction and learning are the crucial goals for these code reviews, never judgment.
Teamwork
I forget where I read this, but the difference between a team and a group is that a group all leaves a meeting at its deadline regardless of where things stand, whereas a team will stay as long as it takes to get things to a satisfactory point.
In practice this makes no sense. Sometimes people really do have to be somewhere at a certain time, and sometimes despite your best efforts, you have to force a less than ideal resolution. But the idea is not to leave things open and unresolved at the end of the day. Make some sort of plan for the next session, or if someone can stay and play cleanup, they should and earn the gratitude of the group. Make sure it's not the same person every week.
Get the entire group involved. If someone's quiet, explicitly invite them to weigh in with their position. Assign homework to the whole group to prepare a decision on items which affect the overall team course. Send around status updates during times when everyone's too busy on different areas to meet.
Code and documentation is communal. If any problem is found, it's your problem to fix, regardless of its origin. No drawing territorial boundary lines, if it's someone else's code, take it as an opportunity to learn about its inner workings.
Let everyone speak their minds, and focus on the ideas, not the person.
Try to be the first to find and acknowledge the weaknesses of your own idea, and the strengths of each opposing idea.
Find common ground. Find and point out things about ideas you disagree that are positive, and be sincere.
If the discussion drags on, put it to a group vote. Once the group decides, that's it, no revisiting the topic until taking action to make things work as they stand.
Able developers seek worthy, difficult projects to grow their skills. Give yourselves a difficult and interesting project, so you'll attract hungry developers, and everyone will rise to meet the challenge.
Read this post-mortem fresh from the source, the project manager on Robot Rock. Robot Rock is an open-source interactive music framework built in 11 weeks by 5 UW undergrads for CSE 403, a software engineering course.
Suggested reading:
The Pragmatic Programmer
How To Win Friends and Influence People
At the beginning
You are the one to set the tone. Work as hard as it takes to get the project off the ground, and demonstrate what your expectations are. The sooner you demonstrate that the project could be a success, the sooner you win the rest of the team's dedicated efforts, and you can't get far without that.
This is the time for experimentation--it will never get easier to test out different concepts!
For example, Robot Rock started with separate demos of drum sounds in a loop and a trivial non-functional UI. Mashing these together was a major milestone and morale boost, demonstrating the fact that all of our technologies can live and work together in the same environment in a doubtful time when we still struggled to set it all up.
It also provided a tangible way to demonstrate that a lot of the hard work was already done, and was useful for settling disputes through experimentation instead of abstract arguments (for example, when designing architecture based on the performance limitations of our libraries).
Establish regular meeting times during the week, and hold onto this inertia. Meeting often is the best thing your group can do.
Don't think of yourself as any sort of authority, think of yourself as the sniffable glue keeping your team happy and productive and it's up to you to figure out exactly how to make that happen! If you read Dune you know that as soon as you start giving orders, people stop acting autonomously, so don’t command, lead instead.
Testing
Write lots of tests in the beginning, but keep them fairly general/flexible since things will be fairly tumultuous at the start and tests will die quick and noisy deaths.
You need the tests for just two purposes: to verify and enlighten. Tests help maintain confidence and momentum, and allow you to breath easy knowing that you aren't breaking anything as you hurtle with breakneck speed through the rapid development which takes place after the design is settled. Of course, with all this code, you need a way to demonstrate what it all does. Tests are a great way to communicate to your team mates how to actually use the code you're writing.
Don't let tests prevent refactoring--accept that you will have to some tedious effort fixing them up. If you hate the thought of this, don't go hog wild writing tests for things likely to change, and write professional tests which don't have lots of repeated code.
Don't leave your tests broken.
Planning
Assembla is a hugely valuable tool. It integrates a ticket system, file hosting, milestone tracking, wiki, and repository with online code browsing.
Use the wiki to maintain your documentation and deliverables. Use the milestone tracking to organize and shuffle your tickets around in time.
As soon as you think of something that needs to be done, create a ticket for it.
Design
So, you're in 403, which means you're writing code for 11 weeks, then throwing it away, right? There's some truth to this, but if you write ugly code you can't use it to convince people to hire you. Often job postings require an example of code you've written to solve a non-trivial problem, and so this is your chance to shine to all future employers.
There are a lot of general things which constitute a good design, such as modularity, simplicity, modules which are decoupled, interfaces, etc.
This isn't really about bad design/good design though. You want a design which will make your high level goals natural to achieve.
For example, we knew at the start that it was critical for Robot Rock's music to feel very responsive and sensitive to user manipulations.
We also knew that we wanted the audio generation to have no influence over the rest of the design, in case we wanted to drastically change the way we generate the actual musical tones.
From these first principles, we could evaluate each potential design (everyone came up with a rough design as a homework assignment) and examine if it allowed for real time responsiveness, as well as decoupled abstract song data from the underlying generation. Once these requirements were met, we went on to examine other more general qualitative characteristics, such as separation of components and simplicity.
If we started with an arbitrary design goal, such as simplicity, we may have locked ourselves into a design where real time responsiveness was anything but natural, despite whatever other elegance is achieved. In other words, we would have picked A Good Design, but it'd be The Wrong Design.
Features
Build the simplest thing that meets the requirements.
If there's a feature you want to add, consult your users first. Make a prototype before investing the time in something which the user may not actually care about.
No optimization until feature complete, or later even.
Understand when code is good enough...I'm just parroting The Pragmatic Programmer at this point, so just read that instead.
Grow Experts
Assign each group member an area of ownership. It doesn't have to be something they are familiar with, just something they care the most about ideally. There should be no part of the project without an associated leader. Everyone can just refer to this person for questions, and it will deepen their understanding of the respective area. It also provides resume bullet points.
Not everyone will have the same knowledge level of the languages you use. It helps to have at least one person experienced with each language utilized, and they should oversee code reviews with the less experienced members, at least during the initial stages. It goes without saying that the instruction and learning are the crucial goals for these code reviews, never judgment.
Teamwork
I forget where I read this, but the difference between a team and a group is that a group all leaves a meeting at its deadline regardless of where things stand, whereas a team will stay as long as it takes to get things to a satisfactory point.
In practice this makes no sense. Sometimes people really do have to be somewhere at a certain time, and sometimes despite your best efforts, you have to force a less than ideal resolution. But the idea is not to leave things open and unresolved at the end of the day. Make some sort of plan for the next session, or if someone can stay and play cleanup, they should and earn the gratitude of the group. Make sure it's not the same person every week.
Get the entire group involved. If someone's quiet, explicitly invite them to weigh in with their position. Assign homework to the whole group to prepare a decision on items which affect the overall team course. Send around status updates during times when everyone's too busy on different areas to meet.
Code and documentation is communal. If any problem is found, it's your problem to fix, regardless of its origin. No drawing territorial boundary lines, if it's someone else's code, take it as an opportunity to learn about its inner workings.
When Conflict Occurs
Let everyone speak their minds, and focus on the ideas, not the person.
Try to be the first to find and acknowledge the weaknesses of your own idea, and the strengths of each opposing idea.
Find common ground. Find and point out things about ideas you disagree that are positive, and be sincere.
If the discussion drags on, put it to a group vote. Once the group decides, that's it, no revisiting the topic until taking action to make things work as they stand.
Think Big
Able developers seek worthy, difficult projects to grow their skills. Give yourselves a difficult and interesting project, so you'll attract hungry developers, and everyone will rise to meet the challenge.
Tuesday, June 2, 2009
How to buy coffee
If you're a consumer in the market for 'specialty' coffee (i.e. anything above Folger's grade) then there are a myriad of options, brands, and considerations designed to address concerns ranging from taste, to environmental impact, and social justice.
Fair Trade is the most well-known label, designed to protect the price points of workers on small coffee farms. Although it guarantees a floor price for the coffee and other protections for the workers, even experts who support Fair Trade, such as Mark Pendergrast, are quick to point out its limitations.
Fair Trade can only offer protections to a small percentage of the workers in the coffee trade since large and medium farms aren't eligible, and quality is left to its purchasers to determine.
Certified organic coffee may seem likely to benefit the health of coffee consumers, but its real impact lies elsewhere. In a speaking engagment at the University of Washington, Pendergrast noted that the pesticides affect the cherry hulls, and don't penetrate to the bean itself. Drinking organic coffee, then, does not necessarily offer a health benefit over traditional methods, but it does benefit the workers who harvest the cherries by hand, and are exposed to the poisons.
Beyond fair trade and organic, there is direct trade, which has no certification, and several bird friendly/shade grown certifications which have numerous variants.
With careful research, you may learn about the organizations and certifications involved for these different brands and labels, or discover a roaster which participates in Direct Trade practices which are agreeable with your purchasing habits.
Yet, for whichever stories come printed on the label or distributed in information on certification practices, however, they simply remain stories. The truth about the chain from the farm to the cup often remains unknowable opaque, and is spun by marketing professionals looking to cash in on the conscientious consumer of specialty roasts.
For a glimpse of what could be the reality of direct trade coffee, for example, look no further than this review of Zoka coffee roasters (my emphasis):
Although I'd like to believe that purchasing specialty coffee with certifications from international organizations is making the world a better place, clearly there is still a certain amount faith involved and the actual results are difficult to know.
My recommendation to the truly conscientious consumer: buy organic Hawaiian coffee. It may be expensive, but:
Otherwise, prepare to invest plenty of time reading literature about different biodiversity certifications or getting to know your local importer better than they may be comfortable with.
Fair Trade is the most well-known label, designed to protect the price points of workers on small coffee farms. Although it guarantees a floor price for the coffee and other protections for the workers, even experts who support Fair Trade, such as Mark Pendergrast, are quick to point out its limitations.
Fair Trade can only offer protections to a small percentage of the workers in the coffee trade since large and medium farms aren't eligible, and quality is left to its purchasers to determine.
Certified organic coffee may seem likely to benefit the health of coffee consumers, but its real impact lies elsewhere. In a speaking engagment at the University of Washington, Pendergrast noted that the pesticides affect the cherry hulls, and don't penetrate to the bean itself. Drinking organic coffee, then, does not necessarily offer a health benefit over traditional methods, but it does benefit the workers who harvest the cherries by hand, and are exposed to the poisons.
Beyond fair trade and organic, there is direct trade, which has no certification, and several bird friendly/shade grown certifications which have numerous variants.
With careful research, you may learn about the organizations and certifications involved for these different brands and labels, or discover a roaster which participates in Direct Trade practices which are agreeable with your purchasing habits.
Yet, for whichever stories come printed on the label or distributed in information on certification practices, however, they simply remain stories. The truth about the chain from the farm to the cup often remains unknowable opaque, and is spun by marketing professionals looking to cash in on the conscientious consumer of specialty roasts.
For a glimpse of what could be the reality of direct trade coffee, for example, look no further than this review of Zoka coffee roasters (my emphasis):
I worked for Zoka for a little over a year and in that time I saw: sexual harassment (from the head of the company which made the girl move out of town), failure to pay coffee farmers, threats from various upper management to the employees (I can't tell you how many times I heard the head of the company say "You have to like me, I'm your boss!", firing people for calling in sick and finally their finniest and latest: embezzlement of employees tips to make up for the money that was lost during a break in. The owner does not care about anything but himself and his money which he has no idea how to manage.
Although I'd like to believe that purchasing specialty coffee with certifications from international organizations is making the world a better place, clearly there is still a certain amount faith involved and the actual results are difficult to know.
My recommendation to the truly conscientious consumer: buy organic Hawaiian coffee. It may be expensive, but:
- You are buying American products, backed by American labor laws.
- It is delicious coffee.
- The farmers are not exposed to harmful pesticides, and neither is the environment.
- You can visit the farm yourself, or see pictures from tourists. I highly recommend visiting, though!
Otherwise, prepare to invest plenty of time reading literature about different biodiversity certifications or getting to know your local importer better than they may be comfortable with.
Tuesday, March 24, 2009
Project Euler
I started working on Project Euler since I had some down time during spring break.
I started working yesterday evening and did some more problems this morning. I've solved the first 10 but still am not even halfway to completing 'Level 1.' The good news is that most of the problems don't take longer than 10 minutes to start to finish, especially when using Python. Generators, list slice assignments and list comprehensions made things like iterating Fibonacci sequences and computing prime numbers fast and easy.
I started working yesterday evening and did some more problems this morning. I've solved the first 10 but still am not even halfway to completing 'Level 1.' The good news is that most of the problems don't take longer than 10 minutes to start to finish, especially when using Python. Generators, list slice assignments and list comprehensions made things like iterating Fibonacci sequences and computing prime numbers fast and easy.
Saturday, March 14, 2009
What I've learned in each course this quarter
Networks:
Now that I've taken Networks, I believe I could describe, at some points in excruciating detail, everything that occurs when a user clicks 'Go' to load a webpage, to the page displaying on their screen.
This includes all the Operating System considerations, sockets, threading, buffering, packets/segments, error detection schemes, framing, Ethernet, DHCP, ARQ, NAT, STUN, TCP, IP, reliability, ordering, congestion control, routing, forwarding, heterogeneity of network topology, BGP; plus real-time application support, wireless protocol, and RFID thrown in just for fun.
Things we didn't learn about: DNS. But I can fill in most of the blanks on this one myself. And we didn't spend much time on application layer protocol such as HTTP, though we touched on SMTP some.
We covered so much in this class that my understanding of network principles didn't congeal until it was time to review for the final.
Algorithms:
We surveyed a suite of useful problems solved using Greedy algorithms, Divide and Conquer techniques, and Dynamic Programming. We also learned how to show that a problem is NP complete or prove that an algorithm is correct and/or efficient.
Key takeaways: the incredibly useful technique of dynamic programming to exploit redundant subproblems to solve a larger problem efficiently, and firm understanding of the definition of NP and NP-complete.
Technical Communication:
This class sharpened my understanding for and gave me practice with presenting technical information. Still, the one liner for this class remains:
Tell them what you're about to tell them, tell them, then tell them what you just told them.
Now that I've taken Networks, I believe I could describe, at some points in excruciating detail, everything that occurs when a user clicks 'Go' to load a webpage, to the page displaying on their screen.
This includes all the Operating System considerations, sockets, threading, buffering, packets/segments, error detection schemes, framing, Ethernet, DHCP, ARQ, NAT, STUN, TCP, IP, reliability, ordering, congestion control, routing, forwarding, heterogeneity of network topology, BGP; plus real-time application support, wireless protocol, and RFID thrown in just for fun.
Things we didn't learn about: DNS. But I can fill in most of the blanks on this one myself. And we didn't spend much time on application layer protocol such as HTTP, though we touched on SMTP some.
We covered so much in this class that my understanding of network principles didn't congeal until it was time to review for the final.
Algorithms:
We surveyed a suite of useful problems solved using Greedy algorithms, Divide and Conquer techniques, and Dynamic Programming. We also learned how to show that a problem is NP complete or prove that an algorithm is correct and/or efficient.
Key takeaways: the incredibly useful technique of dynamic programming to exploit redundant subproblems to solve a larger problem efficiently, and firm understanding of the definition of NP and NP-complete.
Technical Communication:
This class sharpened my understanding for and gave me practice with presenting technical information. Still, the one liner for this class remains:
Tell them what you're about to tell them, tell them, then tell them what you just told them.
Thursday, March 5, 2009
Warfare and network algorithms
How much does computer science owe to the military?
Well let's see...
It's nice to think that most of the early computer science leaps and bounds were born out of the circle surrounding the hippie 60's XEROX PARC culture, but the truth is far more...crew cut.
Well let's see...
- The Department of Defense basically invented the internet, and with it "our entire field" to paraphrase the UW CSE department chair during one of the lectures he gave...on operating systems.
- The Ford-Fulkerson network flow algorithm used to route traffic through nodes was invented in 1956. Not for sending oil along pipelines or routing civilian traffic--to determine if the Russian military could successfully move enough troops into neighboring countries using the roads currently in place to successfully invade them. The Cold War is responsible for so much useful science!
- The limitations of reliable communication with TCP were known long before, based on the notion of two separate armies coordinating an attack on a territory from two fronts. If only one attacks alone, devastating losses are incurred. If both strike simultaneously, the territory is easily conquered. The only problem? Trying to agree on a time to strike, with an unreliable messenger running back and forth "routing" messages between the two on horseback or foot. What protocol would you use?
It's nice to think that most of the early computer science leaps and bounds were born out of the circle surrounding the hippie 60's XEROX PARC culture, but the truth is far more...crew cut.
Tuesday, March 3, 2009
Write-Only Memory
It is real.
Here it is, in all its Wikipedia glory
And here is the April Fool's prank pulled by Signetics:
Here it is, in all its Wikipedia glory
Out of frustration with the long and seemingly useless chain of approvals required of component specifications, during which no actual checking seemed to occur, an engineer at Signetics once created a specification for a write-only memory and included it with a bunch of other specifications to be approved. This inclusion came to the attention of Signetics management only when regular customers started calling and asking for pricing information. Signetics published a corrected edition of the data book and requested the return of the 'erroneous' ones.
And here is the April Fool's prank pulled by Signetics:
Subscribe to:
Posts (Atom)
Posts
