Java Source - last updated 08.13.06

extended.jar [OPTIONAL 07.17.05] is sort of the ClassLoader and search engine parts for standalone testing outside the application.

Combined.zip. The application itself.

Latest Update 12/3/6
Not a lot of progress. But uploading what there is.
Actually built on "ROME, the premier newsfeed parser library for the Java platform"*
- RSS And Atom In Action, David Johnson
At some point I am still planning on including search engine type support for personalization by allowing selection of content of interest. The intent for now is to base that on Lucene.
I also added in some code indicated on the OS/X java-dev list at The Rabbit Hole

Anyhow, its still interesting so I will probably put more time into it.

1.6 SHOCKER!
It appears that as of java version 1.6 the ClassLoader techniques I used to substitute for the standard java.io classes will not work. So TRZ is less aptly named as far as Transparent goes.
Now TRZ won't be based on a hack and will just have to stand on its own merits as a more powerful, flexible and enhanced API for dynamic zip file access
For my IDE code I am looking at using the new 1.6 provided javax.tools classes. This should still allow TRZ to be used with javac to directly compile to and update zip/jar archives. The other choice would be actually modifying the javac jdk code. This seems less a public API and less supported by Sun. The long term best solution I am hoping Sun will provide for me still seems to be officially supported pluggable filesystems. Then I just make TRZ one of those. See the new comments on JSR 203 farther down for where that seems to stand.

Older TRZ related reading
The following was pointed out on the MRJ list, somewhat of interest to me in relation to Transparent Random Zip. Details some of the file concerns for the upcoming Mac OSX. Some of it familiar to most MRJ developers already, from MRJ file handling. Some of it of seemed to parallel concerns I had with TRZ.
The Challenges of Integrating the Unix and Mac OS Environments.
Transparent Random Zip is an already working version of the "bundles" referred to as opaque directories in the document but with the opposite perspective. I prefer to think I am making the file transparent rather than making a directory opaque.
Another interesting page was pointed out on MRJ List [NOTE: That this is now part of JSR 203, see the following]
JSR-000051 new I/O APIs for the Java Platform.

JSR 203
TRZ would benefit from a more 'standard' way of 'plugging' in filesystem's. For java itself pluggable filesystems are now a part of JSR 203 which is slated for the 1.7 Dolphin release. My impression is that it is again the other parts of the API changes included along with it that are of broader interest and which are seen as complicating the implementation and have caused the series of delays. My impression could be wrong, but I've roughed out a pluggable zip filesystem and it wasn't that terribly difficult.
One way or the other, official pluggable filesystems won't be seen in core java until 1.7 at the earliest.

utilsrc.zip download.
utilgl4.zip gl4java related source.

Quaqua

Bouncy Castle

The full download includes plist parsing, double-clicked self extracting jars, stub launching with MRJ Adapter front-end and so on are included in appedit.jar

Application for verifying OS/X java application bundle correctness is AppEditVerify.dmg

Toolkit for doing other things with application bundles. This includes the start at a application to build more complex self-extracting (OS/X) jar files. Accum for now, will probably name it File Drop at some point. It supports the file drag and drop API to a JTree. There is a neon animated image tacked on that is also supposed to accept the file drops but doesn't yet. Trying DnD and doing some kind of neon l&f controls a couple things I've thought about trying in the past. All of this still very simple at this point. Definitely seems to work well enough for wrapping a singe .dmg, see DocumentMonster below as well.

Download
app_tk_SEA.jar

Temporarily trying to keep more current changes here until I get a better idea of cvs updating the SourceForge project

The source is here

Current work in generation of a Cocoa Java type interface - OS/X Terminal usage would be something like...

Update 09.25.05 I probably should mention here that my current JNIDirect project is trying to automatically generate as much of a interface to the Cocoa API's as I can manage. Last changes in brief, handles more non-'int' types better.

Successfully running some Cocoa things off the generated, JNIDirect and jnilib. Bundling it all up in a jar now jnicocoa.jar. That includes a README in the 'jni' folder so more information there now instead of here

The XSLT file ant.xsl that was just mentioned, as .txt.

It links to the documention for the tasks. You can change that URL to local disk in the ant.xsl here...

<!-- Location of ant documentation, change to local disk if you prefer -->
<xsl:variable name="root">
  <!-- <xsl:text>file:///???/apache-ant-1.6.5/docs/manual/</xsl:text> -->
  <xsl:text>http://ant.apache.org/manual/</xsl:text>
</xsl:variable>

If you have any ant build's that this doesn't display correctly or well, let me know and I'll see if I can improve this.

NSAppleScript execution and possibly other Cocoa Java classes should be run on the AppKit 'main' thread. OnMain.dmg provides a small JNI stub to do a callback to java on the main thread that should provide this.

10/28/2006 - More new functionality. Added Some JNI, generated of course, execution of the AppleScript related

10/22/2006 - Added some additional AppleScript testing and a couple execution option switches. More or less documented in the README file.

Download: ls.dmg

NEW - 10/15/2008
The JNIDirect SignatureToApp class has been one of my own favorites going back to when it supplied some enhanced Classical JDirect MRJFileUtil capability. The following is a straight JNI version showing how JNIDirect might still work as a JNI 'code generator'. I have trimmed out some of the unnecessary dependencies and have gotten the size of jnilib plus jar well below 300K. I think theres enough API for this to be reasonable overhead

There is nothing wrong with the OS X 'open' command. But it isn't extensible and it isn't open source. Such as it is, this is those things. It does add a couple bells and whistles already that 'open' doesn't do and more functionality could reasonably easily be added.

But for this one, for now, this is it. Enjoy if you wish.

Download: ls_jni.dmg

Viewer for Mail.app signatures with osascript Runtime exec of AppleScripts and the BouncyCastle API's for handling the digital signatures. I had added this to my Combined application but it probably makes more sense as a standalone demo.

Updates

Changed to use current Bouncy Castle crypto jars and now bundle them in the download.

Fixed incorrect 'return' character included in the AppleScript for mailboxes with multiple accounts.

Fixed name on certificate information display dialog

I Included a launching script with AppleEvent debug environment variables that I didn't get to work right.

Download is now a dmg

Mail.app still doesn't seem very good about showing this information itself. I filed a bug this time (ID 6460099). I also put in that Mail.app seemed to require two accounts to set up a signature. I found decent signature set up directions for Mail.app signing certificates here.

---

AKS and primes - 01/04/05

Previous of the coding blogs. Follow up curiosity to the MersenneTwist related - sort of.

First, since this is supposed to be a blog of sorts, maybe I should have something sort of anecdotal. Although blogs of the diary type, 'today I stubbed my toe' or whatever, usually seem pretty uninteresting to me.

The year is sometime in the very early 70's. I take the old fashioned telephone handset and stick it in the coupler that passed for a modem at the time. My high school math teacher sticks his head in the small closet sized room housing the dialup network equipment, the coupler and a teletype printer is about all of that, that I'm remembering, and asks what I'm doing. I'm not sure if this is simple curiosity or he suspects me of being the first hacker trying to break into the defense department or in some other way up to no good. I tell him just running my (Basic) prime number program.

So, this is one of my first memories of anything 'computer' - with prime numbers being about the first thing I remember that interested me related to computers. So my interest in prime number programs has a little history and some roots anyhow.

Ya know, I'm starting to sound a little like grandpa Simpson on the Simpsons.

Anyhow the main curiosity was could some real prime number crunching with the current state of the JIT be done as quickly as the same algorithm could native compiled. Alas, the answer so far is that compiled is 4x faster at least as far as my first pass programming skill goes. The code is number.jar if of interest.

A few of the related links I came across if any of those of interest

AKS Primes is in P

Primes

Mersenne Primes

For fast square roots

More on elementary functions

Wouldn't be complete without considering rational approximations

Some floating point considerations

Yet more functions

A floating point TechTip

Some of this from a more crypto viewpoint

A presentation of how algorithm efficiency, time, complexity, what have you is measured

The internet is in my opinion a tremendous resource. The truth is out there. Sometimes it takes some sifting, sometimes a fair of amount of slogging through, and sometimes you still aren't sure you found the absolute TRUTH. But there's a lot of good, free info and stuff. Really, in my opinion there should be courses just on researching the internet. As it is I'm remembering some interesting pages that are not included above. How does the expression go that the journey is more interesting than the destination. Tracking down these things is usually like that, you almost always find something at least equally interesting along the way to what you're looking for. Thats part of what these code blogs are supposed to demonstrate I guess.

A new coding blog. I became curious concerning questions of the speed of java.util.Random that came up as a topic on the BouncyCastle mailing list (Sign-up URL below with PDF stuff if interested). Sean Luke, who sometimes frequents the OS X javadev list, had done a java replacement for java.util.Random based on the MersenneTwist random number generator. For my coding I grabbed the version included with ECJ although it can be linked to more directly it seems here?.

Mersenne Twister is not cryptographically secure. (MT is based on a linear recursion. Any pseudorandom number sequence generated by a linear recursion is insecure, since from sufficiently long subsequence of the outputs, one can predict the rest of the outputs.)

To make it secure, you need to use some Secure Hashing Algorithm with MT. For example, you may gather every eight words of outputs, and compress them into one word (thus the length of the output sequence is 1/8 of the original one).

So that is how I proceeded. I decided to first see if native JNI provided any significant performance differences and if performance with a hash included would still be on a par with Random.

Sample numbers arrived at for Sean Lukes 100000000 generated random int's speed test were as follows...

Times in milliseconds, so 19440 is 19.440 seconds. Fast is the everything in JNI version. Cokus is a suggested optimized version from the Mersenne Twist page. Returned state means the 624 int array Mersenne Twist state is returned in one chunk instead of a int at a time. SHA1 w/ state means that first I hash the 624 int array already mentioned and then return the 390 element int array result as one chunk. I translated the BouncyCastle java SHA1Digest class to C for the Sha1 hash.

Something possibly to note would be that unless the state is returned the JNI versions use significantly more time for all those invocations. If state is returned - that is the 624 element int array for the MT state, or the 390 element array for the hashed result - accesses are then done against that until it's used up when another JNI invocation becomes necessary. 'Returned state' thus resulting in considerably fewer JNI invocations, where all that is usually being done is returning an already computed int value. If you get the state all at once you get considerably better time because you are calling JNI considerably fewer times.

The other thing maybe to notice is that if state is returned even with the SHA1 hash the times are faster than Random. This somewhat surprised me but I think may mean this wasn't a totally pointless exercise. Anyhow it isn't complete. I would like to appy some of the standard RNG tests to my hashed results to see if they still seem to be good RNG's. I'd also like to consider what is mentioned on John Savard's page concerning the Mersenne Twister and it's use as a keystream generator for a stream cipher. To throw in one last link on this rather link intensive blog, pLab - a RNG site or so I gather - mentioned AES makes a nice RNG which I find interesting and might look at.

I guess if you're still reading this you might be interested in mt.zip the code

BouncyCastle and iText PDF. Updated 03.01.04. Added PDFSign and related

The topic of signing/verifying PDF documents came up on the BouncyCastle mailing list. It seemed sort of interesting to me so I started seeing what I could do with it. Not as easy as it sounds - for me anyhow. Currently I can verify for the /adbe.pkcs7.detached signatures that are included in PRAcroForms. At least I assume what I have would successfully work for any such setups. Testing is limited so far to only a few signed PDF documents I have found. Left to do, I have verifying if not PRAcroForm included, signing support besides verifying, and support for the /adbe.x509.rsa_sha 'raw' signatures.

The first document I've been testing with is xsignarticle.pdf here. This document uses the PKCS7 detached and is included in the PDF PRAcroForm field. What I have should handle it.

The second document is Adobe provided from here. The income tax form. This I do not handle yet but hopefully will shortly. Assuming theis pae is somewhat current this may actually be assumed to be the current recommended method of handling signatures?

Anyhow, what I have currently is PDFVerify to verify

03.01.04 I have somewhat simplistic signing working. That is I can attach a signature to a document I am creating and PDFVerify will verify the signature as valid. AcrobatReader 6.0 will not. It will indicate the presence of the signature but doesn't show it as default certificate type as it shows it's own signed ones. And it doesn't enable validity checking itself of the signature as it does it's own.

Anyhow the stuff included in as far as I've got...

PDFSign.java.

PdfSignature.java modified version of the iText class.

PdfHexString.java an iText addition of my own for this. These seem to get handled elsewhere correctly without requiring this class but I haven't determined how and it seemed simple enough to add one.

What may also be of interest, located here for the moment until I figure out where to leave them.

I used this tweaked iText PdfLister.java for parsing practice

And this modifed BouncyCastle code, ASN1DumpLabeled.java to try to figure out what I was parsing

Thought there was more. But I guess that's it for now

astro.jar

download

...because 0 lengths crashed it. Might be files with variable bit rates mentioned as not handled somewhere. I put the following MyStation to the same, code base, directory as the sample jar file. I started up iTunes, did an "Open Stream" for http://localhost:8000 and there you go. Sounded as well as iTunes usually does.

There are probably more sophisticated options out there than this by now and it's probably sort of naive of me to think gosh thats kind of cool. But if anyone else has or does do any tinkering with this - I'd be interested in hearing about it.

stackWalker

Test Cases

Finish the year with a little code exercise inspired from this bouncy castle list thread.

Testing showed that it spent most of it's time in the BigInteger prime number constructor. So to improve things you need to improve BigInteger, I've messed with that, not easy. Or change the algorithm, but theres not much algorithm to change. Just the p = 2q + 1 equation - where q is found, and p is determined where both need to be prime. However, it occurred to me the equation could be applied twice. As done above, and then you can take your first prime as p instead of as q and solve in reverse for q = (p - 1) / 2.

Not real complicated math but it does give you another chance at a solution without having to just start over.

For my testing with 1000 trials at key sizes of 512, 768, 1024 and 1280, there were 9 successful p,q's found going forward [p = 2q+1] and 5 found going backward [q = (p-1)/2]. Those results can be seen at pgptester.txt. The test's PGPTester.java source

However, given Gauss's determination that the number of primes up to any given 'n' approaches...

Prime Number Theorem
π(n) = n/log(n)

which means as I understand it that the bigger the keys you try for the sparser the primes are going to be. So a method that increases the number of successes and actually does so by testing against smaller numbers might not be all that bad an idea. Of course the math for primes this size might just be ruling the whole approach out.

A couple URL's were mentioned on the bouncycastle forum that should improve determination of these "safe primes" as they are called.
Safe Prime Generation with a Combined Sieve
and Double-Speed Safe Prime Generation
My idea is mentioned in passing, so I guess I'll have to cancel that patent application. Someones been there and done that.

Epilogue
As it turns out, this idea doesn't work anyhow. The algorithm is designed to start out with the 'q' bit length one less than what the resulting 'p' length should be so that the final 'p' bit length value is correct after the 1 bit left shift that always happens from the multiplication by two in p = 2*q + 1.
Taking the original random value as 'p' for going backward [ q = (p-1)/2 ] means that it's length is one bit too short to start out. Which doesn't work for the exact length required by the ElGamal algorithm.
So not only no patent, but back to the drawing board.
A considerably faster version for bouncycastle may be appearing without the need for it

For more information on using a shell script instead of a JavaApplicationStub for launch, see Greg Guerin's prelude scripts

The test case that I wrote for this is DocumentMonster.dmg This is a self-extracting jar containing the former .dmg file. Works better with my current ISP setup. See the app Toolkit download above for the application that creates the self-extracting jar. It includes some .dropper file extension files for testing by dragging to the application icon. It issues println messages to Console.app indicating success. If you look in the applications Contents/MacOS folder you will see the Prelude script that may be used in place of the JavaApplicationStub. You might want to look at Greg Guerin's information, from above, before getting much farther along with that.

Currently it is set to the script running java directly, where open document events fail condition. So to see it actually succeed in handling open document events you will either need to change the script or Info.plist to run the JavaApplicationStub instead. Of course, then if it doesn't succeed you need to determine if you did this correctly or Launch Services is now not working. More on that in the following

Launch Services Issues: There are other concerns that figure in when you start changing the Info.plist entries or adding and changing scripts within your OS X application bundles. These changes can break Launch Services functionality for your application in different ways.

NOTE 12/12/7 Since this was originally written further experience has led to a better understanding of this issue. This problem seems to mainly occur following some kind of manual update.

e.g. right clicking the application bundle and using Property List Editor to modify the Info.plist file doesn't update Launch Services and can break functionality.

More details on the concern follow.

Some OS X java-dev mail list posts reflecting some of the problems and their work arounds

• http://lists.apple.com/archives/java-dev/2006/Jul/msg00233.html
• From another list http://lists.apple.com/archives/carbon-dev/2006/May/msg00952.html
• This preceded my own bug report, http://lists.apple.com/archives/java-dev/2006/Jan/msg00107.html.
• http://lists.apple.com/archives/Java-dev/2005/Mar/msg00772.html

Its sort of insidious knowing if Launch Services being out of sync is actually your problem, or configuration, or something else. The lsregister command can get you more information but I won't get into that for now. Well, someone did something interesting with that recently, trying to get a list of applications, that I will mention. Actually, for some reason I can't locate my own bug on ADC to provide the number. One other bug was mentioned as being filed on open document issues off the current list thread, so Apple probably remains aware of the issue.

The download
ae_jni.dmg