Re: [AD] about allegro development

[Shawn Hargreaves <shawn@xxxxxxxxxx>]

Peter Wang <tjaden@xxxxxxxxxx> writes:
> I didn't volunteer for this earlier, as I have been quite busy with
> school.  However I believe I have all the required tools (albiet old and
> rusty), so if someone can supply me a Build-A-WIP-HOWTO in time then I
> could probably get 3.9.32 out sometime this weekend.  Does anyone know 
> if Shawn is reading his personal email?

I've been following both that and the list on and off, just not replying 
to any of it for a while :-)

Regarding my personal situation at the moment, I will have some free time 
later next week, so if nobody gets to it before then, I can get a WIP 
done for Wednesday or Thursday. In the longer run I'm not going to be 
totally absent, but I'm not going to have so much time as previously, so 
I won't be able to answer email every evening, or release a WIP every 
fortnight (I'm going into a new job with a full art/design team, but 
where I might be the only coder for a while, so I'll have my work cut out 
for me there :-)

In other words I can do my best to keep things ticking over, but it would 
be better if all you lot didn't have to slow down to match me, so I think 
it would still make sense for other people to take over the driving seat. 
Either you could just take this over and carry on in the same basic way 
that I've been doing things, or it occurs to me that this might be a good 
time to think about moving to a CVS system, so we could have half a dozen 
people with checkin access, and then releasing a WIP would just be a 
matter of pulling out a version, making sure that it worked, and zipping 
it up for distribution. It's really up to you to decide how you'd prefer 
to do this: obviously we need to have some core people checking and 
applying patches from unknown contributors, but if none of you want to 
commit to doing that as a fulltime job, CVS might be a good way of 
spreading the workload. I've never used it in an internet scale project 
before, but it does seem to work well for the people who do...

Regarding building a WIP using the current system, first step is to grab 
all the patches (I've got all the mailing list ones, plus a few that were 
mailed to me privately, sat in my todo folder, and can send a zip of that 
to anyone who wants it).

Probably the hardest bit is deciding what to do next. Some things are 
obvious (bugfixes, new drivers, patches from people who can be trusted to 
know what they are doing), and can just be applied as-is, but others need 
to be examined more carefully. Sometimes the idea is just bad, other 
times the idea is nice but the implementation is bad, so it needs 
rewriting, and other times there are just minor problems like weird code 
layout (run through indent), naming conventions that don't match the rest 
of Allegro, or various subtle things that might not occur to someone less 
familiar with how Allegro works, that need to be checked for (eg. string 
handling that isn't Unicode-safe, various sorts of non-portable 
assumption, etc). So you need to check through the patch, decide what to 
do about it, adjust anything that needs adjusting, and then reply to 
whoever sent it in to explain what you did with it.

Then you have to update the docs: docs/changes._tx for all but very minor 
tweaks, docs/thanks._tx for any new contributors or big bits of code that 
aren't covered by their existing description, docs/allegro._tx for any 
API changes or new hardware drivers, and allegro.cfg for anything that is 
configurable (new driver, config variable, keyboard map, translation, 
etc).

Then it's ready to upload. The first step is:

   misc/fixver.sh 3 9 32 WIP

That adjusts the docs, header, makefiles, etc, with a new version number.

Then I reboot into win95, and run:

   misc/zipup.sh all3932.zip /backup/allegwip/all3931.zip

This will generate the zip and diff files one directory up from the 
current location (obviously replace the second param with wherever you 
have a copy of the previous version, for diff generation). The zipup 
script should work on Linux too, but I run it in win95 because that 
allows it to run fixdll.bat, and to generate dependencies for various 
compilers (although that won't work for you without some special setup: 
see below).

Then I reboot back into Linux, and run a local script of my own, 
scripts/upalleg, which deletes demo.dat from the zip created by zipup.sh 
(to save space), runs misc/mkunixdists.sh to create the .tar.gz and .rpm 
versions, copies everything across into my web directory, and generates a 
new wip.html (I can send you a copy of this if you like, but it's quite 
specific to my system, so you might be as well just running 
mkunixdists.sh directly yourself).

Then finally you get to watch the modem lights blinking for about three 
hours while it uploads everything :-)


George Foot <george.foot@xxxxxxxxxx> writes:
> The other part is creating the WIP package, and that's more complicated 
> because for reasons I don't fully understand, it has to be done from 
> Windows (something to do with generating .def files for the DLL, and 
> dependency information for the various compilers). 

zipup.sh will (or at least should) work anywhere, but before creating the 
zip, it tries to generate compiler dependencies, Windows .def files, and 
to leave the package in djgpp format. The .def generation bit just checks 
whether fixdll.bat is executable, and runs it if so (which succeeds on 
DOS, fails on Linux). For the other compilers, it goes through each in 
turn checking for the executability of eg. /batch/msvc.bat. If that 
exists, it then runs:

   fixmsvc.bat
   /batch/msvc.bat make depend

the idea being that this batch file will set up the environment in a 
suitable way for using MSVC, and then generate dependencies with it. If 
you don't have any /batch/compiler.bat files, you won't get any 
dependencies, which probably isn't the end of the world, but would 
require everyone to run "make depend" themselves before they can build 
the lib, leading to complaints from everyone who doesn't have sed 
installed.

> Ultimately, it looks to me like Shawn's system is nice and convenient, 
> and easy for somebody else to do in his absence, provided they have all 
> the compilers.  Either I'm wrong, or that's a major oversight. If this 
> is the case then it might be worth revising the script to make it 
> possible to run it repeatedly, on different systems, to create all the 
> information properly.

It does occur to me that it should be possible to make zipup.sh work 
entirely in terms of Unix tools, which would be great as it would allow a 
full set of dependencies to be generated on any host platform that has 
bash and gcc (the current system doesn't scale well as the supported 
platforms increase, for instance I have no way to include BeOS 
dependencies in my distributions).

Portability problem areas in zipup.sh are:

- running "make -s veryclean" at the start. This isn't fatal, except that 
  it will fail on Unix unless configure has been run first (as there 
  would be no makefile). I think this can be replaced by a bit of sed and 
  xargs trickery, though, since the list of files to be cleaned is in the 
  three defines at the bottom of makefile.lst, so those names could be 
  extracted directly without requiring the makefile to be used at all.

- running fixdll.bat. I think this can be done without touching 
  command.com, though, since it is basically just a list of echo, sed, 
  and copy commands. A trivial .bat -> .sh converter could be written as 
  a sed script that would delete the "@echo off", kill all rem lines, 
  change copy to cp, and put quotes around the strings being echoed, 
  after which a simple "sh adjusted_fixdll.bat" would do the trick. ie. 
  if zipup.sh contained this .bat -> .sh conversion process, it would 
  then be able to run these simple batch files directly on any Unixlike 
  platform!

- fixmsvc.bat, fixdjgpp.bat, etc, are the same: a bit of sed to convert 
  them to shell script format, define a Unix replacement for utod, and 
  they could be run directly.

- dependency generation is the hairy bit: this is done by "make depend", 
  which runs gcc and then sed (portable enough), but would be using the 
  wrong version of gcc when run on other platforms than the target. For 
  things like the MSVC and Watcom makefiles, it already works around this 
  to some extent by eg. telling gcc to define __WATCOMC__, and undefine 
  __GNUC__, DJGPP, etc (this works because for dependency generation, we 
  only have to preprocess the code rather than actually compiling it, so 
  as long as it can trick Allegro into looking at the right headers, it 
  gets right results even when using the wrong compiler). The only hairy 
  bit, and what stopped me from doing this before, is when it comes to 
  include system headers, and finds that eg. windows.h doesn't exist on 
  Linux. I think this can be fixed with a bit of ingenuity, though: 
  either put empty copies of all those headers somewhere just to stop it 
  complaining, or somehow stop Allegro from even trying to include them 
  when being scanned for dependencies.

So I think this can be done, and would certainly be worth doing (it would 
be a big win to have 100% complete package generation for all target 
platforms, no matter what host platform the scripts are run on). Trying 
to make bash run the batch files is a bit of a hack, and I suppose we 
could just include second copies of all these scripts that are already in 
.sh format, but the batch files are needed for normal DOS/Win users, and 
I hate to duplicate anything unnecessarily...


Hmm, that's a lot of typing without much of a conclusion :-) In the short 
term, things are a bit of a mess with too many bugs hanging out: I'll 
deal with some of the more ambiguous patches (and make zipup.sh more 
portable) later in the week, so I can leave you with a nice clean version 
of everything, but in the meantime I guess it's up to Peter whether you 
want to do a patch-applying WIP this weekend or not. In the longer term 
it's again up to you lot whether one of you wants to take over, or keep 
swapping things around, or maybe switch to a CVS system: either way, 
hopefully we can automate the upload process enough to prevent it being a 
problem, so that the maintenance work will be entirely on the decision 
making and implementation side...


-- 
Shawn Hargreaves - shawn@xxxxxxxxxx - http://www.talula.demon.co.uk/
"A binary is barely software: it's more like hardware on a floppy disk."