MySql 5: Error No. 1045 Access denied for user 'root'@'localhost' (using password: NO)

I have been driven crazy by this issue, trying to get a MySQL 5.0.21 database created from the wizard in WinXP.

The error message was this:
#1045 - Access denied for user 'root'@'localhost' (using password: NO)

It also talks about making sure port 3306 can be opened. In one of my installations, I was using Norton Firewall and I did have to open up port 3306, but I also had this error on a machine without a software firewall and that was not the issue.

The core problem seems to be that even though you supply a root password, it does not get applied. Does this happen for everyone? I don't know. But the "using password: NO" for me suggests this is what was happening.

Solution #1

The solution in that case is to ignore the error about applying security details, log in with an empty string root password, and change the password. MySql bug#6891 tracks this for version 4.1.7. The workaround given there is this (credit Freek Bos):

So when you start the MySQL command line client.
You simply press ENTER when asked for a password.

Now you can change the password by running the following line.
SET PASSWORD FOR 'root'@'localhost' = PASSWORD('MyNewPassword');

Solution #2

I didn't try this route. The solution that worked for me was

1. Uninstall MySQL
2. Erase the MySQL installation directory, as the old database files are left around.
3. Install MySQL.
4. Choose "Standard Configuration" rather than Detailed.

Somebody had suggested (sorry, I've lost the link now) that there was a problem when using the "Detailed" configuration to create your database initially. They said to choose "Standard" initially to create it and then go back and reconfigure it as desired with the Server Instance Config Wizard. I wonder if the Detailed configuration wizard path has the problem of not applying the given password? If I get a chance I'll try these two options in a clean system and update this post if I confirm anything.

Some people run the Server Instance Config Wizard after getting this error once and find that it now can no longer start the service. This happens if the service was actually started the first time. But don't go this route! Re-running the wizard won't solve the password problem. Follow one of the two steps given here instead.

If this was helpful to you, or you have a correction, leave a comment and let me know! It looks like this is a very common error.

Universally useful feature: go to last edit point

The Eclipse editor has a feature, activated by Ctrl-Q, of taking your cursor back to the point in your document where you last made an edit. I find this extremely useful, and started to wonder why I haven't seen it in other editors.

Regardless, I have to file this feature away under the category of "user interface best practices", along with filename completion.

What is a bug?

A nonprogrammer friend asked me "What causes bugs?"

My answer at the time was that they take many forms, and one form is that a program is used in a way that was not anticipated when the program was written.

For example, you write instructions for a blindfolded person seated at a desk how to get up from the desk and leave the office. The instructions work great in the office environment they were written for, but try to apply them to a different office, or even the same office after it's been "reconfigured", and your program might literally send someone running off into the weeds or stepping into an open elevator shaft.

Certainly a bug is some kind of disconnect between expected and actual behavior, so you'd have to talk about requirements. But the more interesting question is how the bug gets introduced -- how the programmer can think the job is done when it's not, and put it into a simple "cocktail party explanation" (as my econ professor used to say).

So, what's your cocktail party answer to "what causes bugs?"

Followup: Checking for a process running

After some very helpful feedback and further work on my own, I'd like to post an improvement to the previous entry "Checking for a process running".

The idea is to get a list of PIDs of an executable Xyz, or other string of interest, suitable for passing to xargs kill.

First, the code.

#!/bin/sh
# Try to guess which ps command we have, ps -f or ps uxww
# Some ps programs accept both uxww and -f, but we prefer uxww.
if ps uxww > /dev/null 2>&1 ; then
  PSOPTS=uxww
else
  PSOPTS=-f
fi
ps -u $USER $PSOPTS | grep [X]yz | awk '{print $2}'

Now, the explanations. Except for #1, these suggestions are from Todd Eigenschink.

1. It's a script, not a one-liner. In my environment I needed to use the command on multiple platforms, and there are two different versions of the ps command on the different platforms. So a script is needed to detect which one is used so the right arguments can be passed. If you know which ps form you need, you can reduce this to a one-liner and perhaps make it an alias.

2. The -u option is supported by both flavors of ps and eliminates a separate grep call to filter out my processes. Actually, even the -u is probably unnecessary as I think ps emits only your own processes by default.

3. The [X]yz pattern is a sneaky way to eliminate the need for a separate grep to filter out the grep process itself. The pattern [X]yz matches Xyz, but now the grep process won't have Xyz in its ps entry, and thus won't include its own process in the output.

4. The awk fragment to extract the PID field from the ps output eliminates the need for the sed and cut commands I posted earlier.

Yacking about a multi-language parser generator

If you've ever written even a toy-sized compiler or interpreter, the parser is one of the first things that slows you down. You have an implementation language in mind, and you cast about for the parser generator. Maybe you're old-school with C/Lex/Yacc, maybe it's Java/JavaCC, or maybe a less mainstream pair like SML/SMLYacc. Maybe you go the route of the self-written recursive descent parser and forego a parser generator. But this choice is a bit like choosing a bank for a CD, because once you have the parser written, you get substantial penalties for early withdrawal from the language and parser implementation choice.

Enter GOLD, a multi-language parser. The concept is dirt simple: generate the DFA state tables and the LR parse table as text-formatted data, and take your lexer and parse tables to any language that has the parsing engine. Even if an engine is not available for your platform, it is not hard to write that part which accepts tokens and consults the parse tables.

The key to this is in questioning the approach of most parser generator systems which put the parsing rules in line with the code that builds the abstract syntax tree. Of course by providing a scheme for building the AST, you've tied yourself to a particular language.

Reuse

By extracting the parse table data to a portable file, GOLD has opened up two different avenues of reuse:

1. GOLD parse tables can be executed in a large number of languages.

Right now, the GOLD page lists at least 7 different languages you can write your interpreter/compiler in while using a GOLD-generated parse table, including C, C++, C#, Java, and Python.

You've already written an interpreter in Java/JavaCC and want to switch to C++? Now you can switch without having to learn Lex/Yacc.

2. Because of the language portability, parsers can be shared more easily

If somebody has already written a Ruby parser or an SQL parser for sharing, you're now probably more willing to pick up the GOLD input and compiled grammar table, because you're free to drop it into the language of your choice. The GOLD site makes available grammars for about 14 major languages including C, LISP, Pascal, Smalltalk IV, and SQL 89.

GOLD as it is now

After all this talk about portability, the bad news is that the current GOLD parser builder is a closed-source Windows-only GUI application. Ouch. While obviously not a theoretical limitation, it's a practical barrier to entry for plenty of would-be adopters who may work only a Unix box or who don't want to run a Windows executable of unknown origin. The builder looks like an IDE, and one possible portable solution would be an Eclipse plug-in. There is also a command-line version of the builder, but for some reason it is also Windows-only at present.

The good news is that the builder looks fairly good. You can regression test your grammar and get fairly good visual feedback on your state tables.

Conclusion

GOLD looks like a great idea in a still-maturing incarnation. I like how it's taken the table-driven notion of parsing one step further. With more community support to make a portable builder to complement the portable tables and engines, I think this tool or something like it will be standard in the years to come.

Classpath not working in Ant junit task

Pop quiz: spot the error in the following Ant fragment:

<path id="common.classpath">
  <!-- some classpath content -->
</path>

<path id="common.test.classpath">
  <path refid="common.classpath"/>
  <pathelement location="${basedir}/lib/junit.jar"/>
  <pathelement location="${basedir}/build/common/test/bin"/>
</path>

<target name="test" depends="compile-tests"
  <junit fork="yes" forkmode="once" printsummary="on">
    <classpath>
      <path refid="${common.test.classpath}"/>
    </classpath>

    <formatter type="plain" />

    <batchtest>
      <fileset dir="common/test/java" includes="**/*.java"/>
    </batchtest>
  </junit>
</target>

You may not even need to know ant syntax to spot the error, since it's an internal inconsistency.

The clue is that you get "class not found" errors for all tests when you run the "test" target.

Give up?

The nested classpath element for "test" is not referring to an existing path ID. I confused an Ant property with an Ant ID. It should say

    <classpath>
      <path refid="common.test.classpath"/>
    </classpath>

without the ${} wrappers that are used to refer to Ant properties. There's a correct path reference in the test classpath definition itself.

Checking for a process running

Recently I've been wanting to know "is process X running? If so what PID or PIDs is it running under?". Running ps piped through grep X is the usual thing. But I got really tired of staring at the extra cruft coming out of ps. I wanted to just see process IDs. So I added some extra filters:

ps auxww | grep $USER | grep X | grep -v grep | sed 's/[[:space:]][[:space:]]*/ /g' | cut -d" " -f2

This returns the PIDs for processes $USER owns with "X" in the ps output. I generally inspect the list and if needed re-run it piped to xargs kill -9:

previous-command-line | xargs kill -9

This works as-is under Linux and places where "ps" supports the "auxww" form which includes the full command line for a process. If probably works as well with the "ps -ef" variant of ps.

The extra parts of the query may deserve some explanation.

 | grep $USER

Filters out processes by $USER (me).

 | grep X

The critical part for filtering the process by command name or other string of interest.

 | grep -v grep

My favorite first optimization: exclude this grep command, I know I don't care about it! Bad of course if X is a string with "grep" in it, or if your username is grep.

 | sed 's/[[:space:]][[:space:]]*/ /g'

This sed command is in preparation for the cut about to take place. It replaces sequences of whitespace with a single space, so that space can be used as a delimiter in the cut command. If you know how to make sed understand [[:space:]]+, let me know.

 | cut -d" " -f2

This extracts the second "field" of each line treated space as the field delimiter.

That's it. If you know of a decent one-liner in Python or Perl or something else, let me know.

Passing system properties from Ant's command line to java or junit

The other day I wanted to do something like this, and have the property passed to a junit task:

% ant -Dcom.blogspot.jfkbits.tmpfile=/tmp/grzlgmpfer-78

My task never saw the property; Ant doesn't automatically pass-through properties. That's fine, it just surprised me, coming from the shell environment variable mindset.

It turns out that sysproperty, syspropertyset, and jvmarg are the only ways to do it.

The jvmarg nested element is the simplest form, as in the example from the Ant documentation:

  <jvmarg value="-Djava.compiler=NONE"/>

If you have a lot of properties this gets tedious.

If the properties you want to pass have a common prefix, as in the example com.blogspot.jfkbits, you can use syspropertyset with the prefix attribute, like this:

<junit>
 <classpath refid="classpath"/>
 <syspropertyset>
   <propertyref prefix="com.blogspot.jfkbits" />
 </syspropertyset>
 <!-- tests go here -->
</junit>

---

Update 17 July 2006:Added text for jvmarg, since that is a way to pass system properties that I previously omitted, and tweaked the description of how to use syspropertyset.

Softcivil engineering

In his recent essay "Make Vendors Liable for Bugs", Bruce Schneier argues for software liabilities to help increase computer security by aligning the industry's capability for improving security with an interest in doing so. That is, the industry lacks a compelling interest to improve security, but it has the capability to improve, so let's make them interested by holding them liable for security issues. I'd like to apply this argument to the tension between software quality and time-to-market. To help gain insights we'll compare software development with other technical fields.

"Towers are popular in Japan: You ride to the top, look down at the city far below you, wonder if the tower is constructed according to sound engineering principles, then ride the elevator back down as soon as possible. That's what I do, anyway." -- Dave Barry, Dave Barry Does Japan

Civil engineering projects, as in Dave Barry's example, often have a public safety aspect that many software projects don't evidence. Some software is of course used in ways that obviously require a high degree of quality and it is engineered well: flight control software, medical diagnostic equipment, and public safety communications equipment. But a lot of software, and I think particularly of web commerce, has a lot of public trust bound up in it, with a marked lack of quality.

What if civil engineering were as easy to do as software? What if an engineer could sit down at his workstation, bang out an XML document describing a hierarchy of bridge members, hit a key, and suddenly a bridge is constructed over the nearby test river? The engineer and twenty of his testing friends drive freight trucks over it, iron out some vibration problems, and their manager says "ship it." Even with more extensive testing, that quick turnaround from a conceptual design to reality would say "book the customers now! more testing just increases cost and drives profits down!" a lot louder than "but it might not work all the time in the next 50 years when it will be used."

Public safety concerns aside, one reason civil engineering isn't done this way is the cost of construction. There's no "XmlToBridge" converter. That, and no chance for a version 2.0, a chance to build a basic bridge and then replace it with a refined bridge with more features in a year or two. Knowing this up front helps everyone involved in the design phase accept that the design phase is critical. The construction cost also gives a quantitative guide or bound on how long is acceptable for the design phase.

In software, the cost of production is the cost of the build process, which could be a few hours for a large product, or in the case of a hosted web application written in an interpreted language, perhaps virtually nothing. This low cost of production and copying is part of what makes software, and computers in general, practical and useful. And cheap, compared with any kind of "hardware" alternative. But if we're trying to align capability for good quality with interest in good quality, the cost of production is working against us.

Hence my claim is that one of software's key advantages, rapid change, is one of the reasons software quality is poor. It lowers the apparent cost of production to a point where it's no longer a barrier that motivates you to double-check your work. Internally a software team can recognize this and take action. Maybe we need to revert to punch-card programming. Rather than trending towards continuous integration (an automated edit-compile-test system), maybe we need a mandatory waiting period between design and coding. I'm not sure what would be effective. But any proposal you come up with to increase quality by investing more time is going directly to the bottom line of software vendors by increasing that production cost. That's why if we want higher quality, we've got to quantify the costs of lower quality, and transfer some of those costs to the software vendors to make it easier for them to decide that quality is worth it.

Brought to you by the letter "J"

Java has been a great boon to the popularity of the letter "J". Today I noticed a new angle on this.

In Firefox, I keep a "Reference" folder in my Bookmarks toolbar (see picture). When I added a Python link recently, I was struck by how "early" in the alphabet "P" was to be the last entry in the list. I did a double-take on what was in my list.

It's a small list, not a good sample size to draw conclusion from. Still it's funny that 8 of the 19 entries start with "J".

A quick survey of hits from Google of "english initial letter frequency" suggests that the most common letters starting English words are T,O,A,W,B,C,D,S,F,M, and R (different pages give different lists). The letter J is of course always near the bottom.

It would be interesting to see an analysis of initial letter frequency in technology names and see if there are other deviances from the standard English distribution.

Ant, the Portable Shell

Today I was considering whether to include Ant in a Java webapp, not because I wanted to run builds inside my web server, but because I wanted Ant's portable tasks like copy and replace. The Ant welcome page says this about itself:

Make-like tools are inherently shell-based -- they evaluate a set of dependencies, then execute commands not unlike what you would issue in a shell...Ant is different. Instead of a model where it is extended with shell-based commands, Ant is extended using Java classes.

Actually, it seems to me Ant hasn't gotten us away from the shell, it's provided us a new shell with portable commands. Granted there are more commands one could want but I'm pretty happy with what we've got.

Come to think of it, generating Java source and compiling it inside a webapp might be kind of interesting...

Language-Driven Applications and the Command Pattern

When you go to write your next great application, consider imitating the work of CPU architects, where you design an Instruction Set Architecture (ISA) and programmer's register model. It would look something like this. Your application state needs to be clearly modeled, akin to the register and memory model of a CPU. Then the application operations would be distilled to a collection of commands, like machine instruction opcodes, which transform the data state. Then you provide a text interface to that command interface, like an assembly language. With this investment, consider where you'd be. First, by focusing on the application core and providing a clearly-defined way to manipulate its state, you get encapsulation from your user interface. Then, especially with the language interface, you allow extension of your application programmatically, third parties can script extensions in ways you never could have anticipated. And finally you can more easily employ the command pattern in your user interface or supporting environment to achieve features like undo/redo and macro record/playback. Do the hard work to make your application language-based, employ the command pattern, and I think you're digging a very solid foundation for a powerful application.

This language-based application is adopted by two notable applications, Emacs and Maya. Emacs, the text editor that doubles as a programming environment and mail reader, is based on Emacs Lisp, and the user interface is built around that language. Any user interface action, up to and including moving the cursor one character left or right, can be repeated by invoking the right procedure in the command window. Maya is the powerful 3D animation package that made the Lord of the Rings movie magic happen. Maya has a stunning user interface, beautiful in the way it makes users productive in an extremely complex environment. However, that interface is on top of MEL, the Maya Embedded Language, which the Lord of the Rings production team used to script "an enormous scene management system".

One benefit of the language based application is of course encapsulation. I suspect a large number of applications have major problems porting to new platforms because the application core and the user interface are too intimately tied. Mathematica, which actually is a programming language application, also happens to have an interesting and useful user interface, which handles things like typesetting and graph plotting. It goes to the extreme in encapsulation, where the user interface and command processor are actually separate processes. The command processor, or kernel, can run standalone and provides a simple text interface. There is in fact an entire transaction language (part of MathLink), for the read-eval-print loop interaction between the kernel and "the front end". It's an open standard, allowing potentially any number of completely different front ends to be built both proprietary and by third parties.

Another benefit of the language based application is future extension. Although there is the possibility of direct support for writing new commands in the language itself, exposing your application's core engine as accessible with a text-based script of commands allows people to write programs in their language of choice that generate "source code" for your application language. If you use Emacs you're probably familiar with the various modes that help you edit a particular file format or programming language, and many of these modes were not written by the same folks who wrote the Emacs core.

The command pattern opens up another handful of interesting features for your application. If your application core already has a suite of commands, they can easily be turned into function or command objects if they aren't already, each of which transforms the application from one state to another.

Probably one of the most useful features available with the command pattern is undo/redo. You keep a "command history" of the state transformations done in your application core. Undo takes the most recent entry from the command history, invokes the undo transformation, and places the entry onto the redo stack. Redo takes the most recent entry from the redo stack and invokes the original command that made the state transformation. The undo functionality may not come for free, but you may find that it's not that difficult and actually rounds out your command language. If you have an "add widget" command, you may not have noticed that your command language lacks the "remove widget" command until you need to support undo for "add widget." In other words, undo/redo commands may simply be pairs of complementary commands in your application language.

By persisting the command history entries to a file with atomic writes, you get a database-style log file. This permits you to recover from a process or machine crash, where you can start with whatever initial state is stored on disk, and replay the log to get back to the most recent log entry that made it to disk.

If you've already got a command history, you can give your application user "macros", in the style of Emacs, where the user designates points to start and stop recording user interface actions, and the resulting sequence of underlying commands can be replayed as an aggregate state transformation. It is in effect a user-defined program in your application's language. Many applications support some form of this feature, including the ability to edit the generated code after the fact.

By combining some of these notions, you get another wrinkle: incremental version control. If you're already keeping a persistent command history log, the user can specify a desired "save point" as the next version of the application state, complete with comments. Now you can provide a feature for a user to roll back to a previous application state, or even branch from a previously saved version into a new direction. This may suit your application well, if the sum of the log entries between two different versions is smaller than saving an extra complete copy of the application state, and undoing or replying those changes is not unduly time-intensive.

So, those are some thoughts I had about structuring applications. Make a clean internal design separated from the user interface and provide a text-based language interface to it.

Idea: Google search annotations and Google Co-Op

I just had an idea, and searched looked up "Google search annotations" to find prior art. Lo and behold, the relatively new Google Co-Op has it. I can't tell if this idea is the same, or if it could be implemented with Co-op, so I'll introduce it and then discuss overlap.

The basis for the idea is the following observation: when reporting a computer problem that could remotely be considered to happen to someone else, the first thing people have been asking recently is "did you search google? what did you find?".

Based on that observation, I started to wonder what if you could take the Google search results, or at least the pages that you looked at, and edit them? You could annotate the links with comments like "this link had a good example". You could even "delete" links as irrelevant to your search, although I imagine if you gave it to someone else they'd want to look through your "recycle bin" of deleted links just in case. The edits could be done on Google's servers, or somehow you could own the data yourself --- write an application or browser plugin which would copy the downloaded search page, parse it, let you edit it, and then let you send around the annotated search results.

It looks like Google Co-Op could support something like that, because it supports adding labels and comments to search results. But it also looks like it's geared for public dissemination of the results and for the tagging to be done by experts. In what I'm picturing, you would encourage people inside your company or organization to do this, and you wouldn't necessarily want those search results to be public.