Showing posts with label ruby. Show all posts
Showing posts with label ruby. Show all posts

Friday, January 04, 2008

Subtleties of Instance Variable Initialization

One of the more mundane corners of programming with objects is the constructor, in particular the initializing of instance variables. In this post we'll find that behind this pedestrian feature are some subtle questions to be answered by implementers.

Consider the following Java class:

public class R
{
public int x = 1;
public int y = x+1;
}
Pretty clear that x is 1 and y is 2, right? What if you switch the order of the declarations?
public class R
{
public int y = x+1;
public int x = 1;
}
This one won't compile. Pretty clear that the intent is the same, x=1 and y=2, but Java won't allow forward references to data members when initializing. Why?

The short answer is because Java evaluates the instance initialization in a certain order. But why is that?

Before answering that, let's compare this example with a couple of dynamic languages. First, Javascript:
// Construct an object with two different orderings depending on its argument
function DynamicDependence(state)
{
return {
x : (state? 1 : this.y-1),
y : (state? this.x+1 : 2)
}
}
var dd = new DynamicDependence(true)
document.write('x='+dd.x+' y='+dd.y+'<br/>')
var dd2 = new DynamicDependence(false)
document.write('x='+dd2.x+' y='+dd2.y+'<br/>')
You'll get this output:

x=1 y=NaN
x=NaN y=2
Javascript is cool with this in the sense that we get no errors. But we do get NaN, not just for the first example (x=1, then y=x+1) but for both orderings.

Can you begin to spot some subtleties in the way objects are constructed? There's something going on, and it happens when there's a dependence between the instance variables. Let's look at Ruby:

class DynamicDependence
attr_accessor :x, :y
def initialize()
@x = $state? 1 : @y-1
@y = $state? @x+1 : 2
end
def to_s()
"{x=#{@x}, y=#{@y}}\n"
end
end

$state = true
dd = DynamicDependence.new
puts dd
# Emits {x=1,y=2}

$state = false
dd2 = DynamicDependence.new
puts dd2
# DynamicDependence.rb:4:in `initialize': undefined method `-' for nil:NilClass (NoMethodError)
In Ruby, the first ordering works, as it did for Java, and the second ordering fails.

The nice thing, for clarity's sake here, about Ruby is that it doesn't offer the option of "implicit initialization" that Java does -- you spell out all initialization in a sequence determined by you in the constructor. The convenient initializers in Java and Javascript have to be done some algorithmic way, which is nice unless the algorithm isn't in the order you particularly wanted. When spelled out as it is in Ruby, we can see that the question of what happens when initializing these members depends on how the "uninitialized" instance variable is viewed by the language. Notice I didn't try this example in C++.

Let's now return to the question of why order of initialization matters. After all, you and I can figure out that when stating "mathematically" that x=1 and y=x+1 it doesn't matter what order you specify these, it means that x=1 and y=2.

What if we wrote the original example this way instead:

public class R
{
public int x() {return 1;}
public int y() {return x()+1;}
}
Suddenly we get the behavior we want, and we can switch the order of declaration. We can declare the y method first, and we get no problems when evaluating x or y.

The trick to all of this is that Java, Javascript, Ruby and virtually every other language out there uses call-by-value, or eager, evaluation of the expressions supplied to the instance initialization. The reason an order needs to be imposed is that in order to supply a value for a field (instance variable), the language needs to evaluate the expression supplied, and it needs to do it at construction time, not later when it's actually used.

Even initializing fields in a certain order is not enough to prohibit problems. For example, we can fool Java's forward declaration checker:

public class BreakJava implements BreakJavaInterface
{
public int x = wreck(this);
// Halting Problem =>
// compiler can't in general tell wreck(this) refers to y
// so this call is allowed even though it will cause problems
public int y = 2;
public int gety() {return y;}

private static int wreck(BreakJavaInterface b)
{
return 2 / b.gety(); // div-by-zero exception at runtime
}

public static void main(String[] args)
{
BreakJava bj = new BreakJava(); // div-by-zero
}
}

interface BreakJavaInterface
{
abstract int gety();
}
Voila, the foolish programmer was able to hang himself yet again.

I know, you've been thinking "who cares" about these funny examples, right? You care if you're designing or implementing an object-based language (with call-by-value evaluation), because you need to think what happens when initializing the instance variables. Maybe you say "fine, I'm implementing Ruby, where I don't need to support an instance initialization syntax, and the usual variable assignment work gets me the desired behavior for free." Fine, the decision is up to you, but there is a decision to be made, because these subtle differences between languages didn't occur by accident. Even so, I can give you a perfectly natural, non-contrived example of instance initialization with dependences:

// Dirt simple Javascript logger
function Logger(name)
{
return {
ERROR : 1,
WARN : 2,
DEBUG : 3,
TRACE : 4,
id : name,
level : this.DEBUG,
log : function(lvl,msg) {
if(lvl > this.level)
return
document.write(this.id+' '+this.level+' '+msg)
}
}
}
var logger = new Logger('example')
We would love to define these constants ERROR, WARN, etc. that go with the class, and we'd also like to initialize this.level to DEBUG. But in this case, level is going to be undefined even though it looks like it could work.

There's some deeper magic going on here that I hope to explore next time. In the meantime, ask yourself what rules you would like there to be for instance variable (non-method) initialization, and how you would implement them.

Monday, February 26, 2007

Parsing Windows SDDL

D:(A;;CCLCSWLOCRRC;;;AU)(A;;CCLCSWRPLOCRRC;;;PU)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;BA)(A;;CCLCSWRPWPDTLOCRRC;;;SY)
You heard me. Yeah, I'm talkin' to YOU!

Well, that's how the computer made me feel when I got that taste of Windows SDDL.

After blustering about doing Princeton CS assignments in Ruby last Thursday, and then considering some of the suggestions from reader comments, I found another thing to try. It's like in Alice's Restaurant when he thinks about what could happen to him, "but when we got to the police officer's station there was a third possibility that we hadn't even counted upon, and we was both immediately arrested."

Well friends, that third possibility of what to write in Ruby is parsing Windows Security Descriptor Definition Language. It turns out that despite all the whining about Windows security, Microsoft hasn't been idle, they have at least written a large number of words on the topic, and their tools generate large numbers of letters, numbers and symbols when you ask them what somebody can do with a given operating system object.

This idea of an SDDL parser was a good candidate for a first Ruby program, because it was a real application. I know how to read basic Unix permission masks, but not this line noise.

When I found this SDDL reference, I was very grateful for the reference, but thought "a program ought to be able to decipher this stuff and print a more readable version".

Well, that became my first useful (ahem) Ruby program. The program in question queries for the security descriptor of a Windows service and then prints a more readable form of the SDDL string.

First impressions of working with Ruby



The first challenge was finding documentation that I could work with. With Python, I needed a "hello world", something a little more advanced to orient me to variable and control structure syntax, and a library reference. I got started a bit with the stuff at Ruby's documentation site like Ruby in Twenty Minutes and Ruby From Other Languages. I ended up mostly using the docs that came with the Windows installer. The Ruby User's Guide fulfilled the first two requirements, and the RubyBook help (Windows help file) was a decent library reference.

This exercise helped me use classes, hashes, iterators, and of course string and array operations.



  • Pro: Control structure syntax is nice, the lack of parens in if, while, and for conditions is a refreshing economy, the regularity of end is easy to comprehend (thank you, Ada, though it doesn't make you tell it what's ending)


  • Surprise: No autoincrement (r++ must be written r+=1)


  • Pro: not having to give an empty tuple for methods that take no arguments (obj.peform() can be written obj.perform)


  • Pro: error messages were quite reasonable, including not only line numbers of the problem area but also those of parts of the call chain.


  • Con: the presence of map and similar iterators is encouraging, but if there's a way to do map that's not a method call, I don't know about it. Talk about a Kingdom of Nouns, that's just what pure object oriented will get you.


  • Con: also in the way of iterators, the parser had a need in a couple of places to iterate over a list, but in a way that map and each can't handle because handling one item required handling the next item in the list at the same time. There didn't seem to be a good way around this, e.g. pattern matching.




Let's examine the the last point. The first example of what I mean takes place at the top of the structure. At the highest level you have productions like this:

SectionList => Section | Section SectionList
Section => "D:" ACL | "S:" SACL | "O:" SID | "G:" SID

We handle this by splitting the string with a ":" separator. Then it's a bit like command-line argument processing. Based on the next token you see, you may need to consume another token following. In this case, it would be sufficient to generate an iterator that gets every other index. It seems like there should be a fairly simple way to do that, since there are iterators over indices, but I couldn't find it yet. A pattern matching facility, as in SML or Mathematica, would be the most direct way to express this though.

Things like the delete_if and compact methods on arrays are nice to have built in, but the list of similar iterating methods didn't feel very complete. For example, I was surprised there didn't seem to be any kind of filtering methods, e.g. return a new array of elements that satisfy this predicate. The compact method is just a special case of such a filtering method. I suppose you could compose collect and compact to get a similar effect, where compact would nil out elements that don't match, but that's a little weak.

I also wonder how to do something like fold. Do Ruby programmers just rely on doing that procedurally with each and accumulating into a local variable?

Well, that's the first foray into Ruby. For the dedicated, I've included the actual program output and source code.




Appendix: The output


Back to post | See source


$ /c/Apps/ruby/bin/ruby ssdlparse.rb eventlog
Security descriptor for service eventlog:
D:(A;;CCLCSWLOCRRC;;;AU)(A;;CCLCSWRPLOCRRC;;;PU)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;BA)(A;;CCLCSWRPWPDTLOCRRC;;;SY)

Discretionary Access Control List (DACL):
Access Control Entry: ACCESS ALLOWED
Trustee: Authenticated users
Directory Service Access Rights:
Create All Child Objects
List Contents
All Validated Writes
List Object
All Extended Rights
Read Permissions
Access Control Entry: ACCESS ALLOWED
Trustee: Power users
Directory Service Access Rights:
Create All Child Objects
List Contents
All Validated Writes
Read All Properties
List Object
All Extended Rights
Read Permissions
Access Control Entry: ACCESS ALLOWED
Trustee: Built-in administrators
Directory Service Access Rights:
Create All Child Objects
Delete All Child Objects
List Contents
All Validated Writes
Read All Properties
Write All Properties
Delete Subtree
List Object
All Extended Rights
Delete
Read Permissions
Modify Permissions
Modify Owner
Access Control Entry: ACCESS ALLOWED
Trustee: Local system
Directory Service Access Rights:
Create All Child Objects
List Contents
All Validated Writes
Read All Properties
Write All Properties
Delete Subtree
List Object
All Extended Rights
Read Permissions


The source code for sddlparse.rb


Back to post | See output


def main
service = if ARGV.length == 0 then 'eventlog' else ARGV[0] end
rawscoutput=`sc sdshow #{service}`
scoutput = rawscoutput.strip()
puts "Security descriptor for service #{service}:"
puts scoutput
puts

parts = scoutput.split(':')
i=0
while i < parts.length
elt = parts[i]
case elt
when "D"
# Get the ACL, a pattern of one or more ACEs like '(A;;CCLCSWLOCRRC;;;AU)'
i+=1 # no i++?
elt = parts[i]
dacl = Acl.new("Discretionary Access Control List (DACL)",elt)
dacl.parse
# TODO add cases for SACL, primary group, and owner
when true
print "Unhandled case '"+elt+"'\n"
end
i+=1
end
end

class Acl
def initialize(name,content)
@name=name
@content=content
end

def parse
puts @name+":"
# contents are one or more '(A;;CCLCSWLOCRRC;;;AU)'
elts=@content.split('(')
# First element is empty string, others have trailing ')'
elts.delete_if { |x| x.length==0 }
elts.map! { |x| x.delete ')' }

@aces = elts.map { |x| Ace.new(x) }
@aces.map { |x| x.parse }
end
end

class Ace
def initialize(content)
@content=content
end

# @content is like 'A;;CCLCSWLOCRRC;;;AU'
def parse
elts = @content.split ';'
@type = elts[0]
@flags = parseOpcodes(elts[1])
@permissions = parseOpcodes(elts[2])
@objtype = elts[3]
@inheritedobjtype = elts[4]
@trustee = elts[5]

# Emit stuff

indent=" "
puts "Access Control Entry: "+AceType[@type]
if @objtype.length > 0
puts indent+"Object type: "+@objtype
end
if @inheritedobjtype.length > 0
puts indent+"Inherited object type: "+@inheritedobjtype
end
puts indent+"Trustee: "+Trustees[@trustee]
parseOpcodesSection(AceFlags, "Flags: ", indent)

parseOpcodesSection(GenericAccessRights,
"Generic Access Rights:",
indent)

parseOpcodesSection(DirectoryServiceAccessRights,
"Directory Service Access Rights:",
indent)

parseOpcodesSection(FileAccessRights,
"File Access Rights:",
indent)

parseOpcodesSection(RegistryKeyAccessRights,
"Registry Key Access Rights:",
indent)

end

def parseOpcodesSection(nameMap, title, indent)
rights = @permissions.collect { |x| nameMap[x] }
rights.compact!
if rights.length > 0
puts indent+title
rights.each { |x| puts indent*2+x }
end
end

def parseOpcodes(opcodes)
# Isn't there a map-like way to do this?!
retval = []
i=0
while i < opcodes.length
retval.push(opcodes[i,2])
i += 2
end
retval
end

end

# The information for the following hashes is taken from
# http://www.washington.edu/computing/support/windows/UWdomains/SDDL.html
AceType =
{
"A" => "ACCESS ALLOWED",
"D" => "ACCESS DENIED",
"OA" => "OBJECT ACCESS ALLOWED: ONLY APPLIES TO A SUBSET OF THE OBJECT(S).",
"OD" => "OBJECT ACCESS DENIED: ONLY APPLIES TO A SUBSET OF THE OBJECT(S).",
"AU" => "SYSTEM AUDIT",
"AL" => "SYSTEM ALARM",
"OU" => "OBJECT SYSTEM AUDIT",
"OL" => "OBJECT SYSTEM ALARM"
}

AceFlags = {
"CI" => "CONTAINER INHERIT: Child objects that are containers, such as directories, inherit the ACE as an explicit ACE.",
"OI" => "OBJECT INHERIT: Child objects that are not containers inherit the ACE as an explicit ACE.",
"NP" => "NO PROPAGATE: ONLY IMMEDIATE CHILDREN INHERIT THIS ACE.",
"IO" => "INHERITANCE ONLY: ACE DOESN'T APPLY TO THIS OBJECT, BUT MAY AFFECT CHILDREN VIA INHERITANCE.",
"ID" => "ACE IS INHERITED",
"SA" => "SUCCESSFUL ACCESS AUDIT",
"FA" => "FAILED ACCESS AUDIT"
}

GenericAccessRights = {
"GA" => "GENERIC ALL",
"GR" => "GENERIC READ",
"GW" => "GENERIC WRITE",
"GX" => "GENERIC EXECUTE"
}

DirectoryServiceAccessRights = {
"RC" => "Read Permissions",
"SD" => "Delete",
"WD" => "Modify Permissions",
"WO" => "Modify Owner",
"RP" => "Read All Properties",
"WP" => "Write All Properties",
"CC" => "Create All Child Objects",
"DC" => "Delete All Child Objects",
"LC" => "List Contents",
"SW" => "All Validated Writes",
"LO" => "List Object",
"DT" => "Delete Subtree",
"CR" => "All Extended Rights"
}

FileAccessRights = {
"FA" => "FILE ALL ACCESS",
"FR" => "FILE GENERIC READ",
"FW" => "FILE GENERIC WRITE",
"FX" => "FILE GENERIC EXECUTE"
}

RegistryKeyAccessRights = {
"KA" => "KEY ALL ACCESS",
"KR" => "KEY READ",
"KW" => "KEY WRITE",
"KX" => "KEY EXECUTE"
}

Trustees = {
"AO" => "Account operators",
"RU" => "Alias to allow previous Windows 2000",
"AN" => "Anonymous logon",
"AU" => "Authenticated users",
"BA" => "Built-in administrators",
"BG" => "Built-in guests",
"BO" => "Backup operators",
"BU" => "Built-in users",
"CA" => "Certificate server administrators",
"CG" => "Creator group",
"CO" => "Creator owner",
"DA" => "Domain administrators",
"DC" => "Domain computers",
"DD" => "Domain controllers",
"DG" => "Domain guests",
"DU" => "Domain users",
"EA" => "Enterprise administrators",
"ED" => "Enterprise domain controllers",
"WD" => "Everyone",
"PA" => "Group Policy administrators",
"IU" => "Interactively logged-on user",
"LA" => "Local administrator",
"LG" => "Local guest",
"LS" => "Local service account",
"SY" => "Local system",
"NU" => "Network logon user",
"NO" => "Network configuration operators",
"NS" => "Network service account",
"PO" => "Printer operators",
"PS" => "Personal self",
"PU" => "Power users",
"RS" => "RAS servers group",
"RD" => "Terminal server users",
"RE" => "Replicator",
"RC" => "Restricted code",
"SA" => "Schema administrators",
"SO" => "Server operators",
"SU" => "Service logon user"
}

##############################################################################
main

Thursday, February 22, 2007

Creative Programming

In some scattered spare moments I've been reading about Ruby, and starting to think about programming in it.

This is a recent goal, mentioned in All I want for Christmas. I've had the motive but not much opportunity for diving in to see what all the fuss is about. Fuss? Well, yes. Ruby is riding at the #10 spot in the TIOBE index of programming language popularity.

As I say, I haven't had the opportunity. This has to do with my house, where I'm essentially rebuilding the living room. Since I'm doing the work myself and with the help of friends, it's not the money pit exactly, but it is the time pit. I guess a well-known equation would say they're the same thing.

Anyway, I'd like to start writing programs in Ruby, but I'm stuck: what do I write? When one starts learning language after language, it'd be useful to have a stock of small but interesting or instructive applications to implement.

One interesting resource I found comes from the I'm Feeling Lucky link result from Google for "creative programming". You know, like "creative writing", only the writing is programming.

That link is Creative Programming Assignments from Princeton's CS department. Robert Sedgewick and others have put together a decent list of assignments for an "Introduction to Programming" course. They are relatively simple programs appropriate to the level, but still interesting, and best of all they cover a range of programming arenas, from algorithms to computer architecture. Here's a sampling of the titles:


  • Digital Signal Processing - Generate sound waves, apply an echo filter to an MP3 file, and plot the waves.

  • N-Body Simulation - Simulate the motion of N bodies, mutually affected by gravitational forces, in a two dimensional space.

  • Particle Collision Simulation - Simulate the motion of N colliding particles according to the laws of elastic collision. (Concepts: priority queue, event-driven simulation)


The N-Body simulation sounds a lot like the gravitational simulation Russ Olsen mentions in his post For the Joy of It, with a similar intent to mine, writing a familiar algorithm in a new language for fun.

So, as I get a few more spare moments hopefully I'll be able to try this experiment of doing some of the creative course assignments in Ruby. I guess I'll have to be careful about posting full source code, in case they want to re-target the course from Java (currently the #1 language in that TIOBE index) to Ruby in a few years.