A Net Yaroze Postmortem and gamedev lessons learnt
In 2016, I spent most of my spare time hacking away with the old playstation, Net Yaroze.
You can see all my Net Yaroze blog posts here.
My previous experience with the Net Yaroze was back in 1999.
I bought it after seeing a UK OPSM (Official PlayStation Magazine) demo disc video saying it was discounted.
At the time I was a IT student (Computer Science), I started in software development, programming C, C++, win32/MFC/OpenGL and Java.
But I majored in information systems (Enterprise), in the mid/late 1990’s business ERP products were all the rage!
It took me about a year to go through the manuals before actually starting anything substantial.
On my Christmas break of 2000 for 3 months, I remember being locked away in my room and crunching so hard, I was actually looking forward to the start of uni!
I learnt how to master the clunky 3D formats and made a 3D side scroller demo, with animated block people, very much like minecraft.
After that experience, I remember thinking, “if game development is this hard, I don’t want to do it!”.
In hindsight, what made it hard was lack of any planning and wanting to do everything myself, I plan more but still want to do everything myself.
Also 2000-1 wasn’t a good time for IT work, with the dot com crash, I remember hearing about a few local game companies going bust.
So, I chose the enterprise/database field instead.
I moved to a bigger city, got a better job, bought an old house and I literally renovated everything myself that wasn’t structural! (yes even water, gas and electricity ).
Fast forward to 2013 and with the major renovations finished, I had spare time again and the Ouya happened.
But I was very disappointed with the cheap hardware and the scandals of “free the game fund”, so I never bought one.
Instead I got a Madcatz MOJO, it was the best Android TV box (’microconsole’ as they were called) and to me it was the Nvidia Shield TV of the time!
So that sparked an interest in game development again, I looked into open source engines, I settled with Gameplay3D(C++) for a while, but the creators removed older OS support which is obviously wrong!
So, next I bought Esenthel(C#) rapid development but the owner decided to put always online DRM.. pfft, no thanks!
Then, I moved to Urho3D(C++), a lot of features but I didn’t like the C++ template design.
By this stage it was late 2015 and I didn’t have anything done, just a few tech demos and the youtube videos that I randomly uploaded of them.
Around this time, the retro gaming and ‘lets play’ (aka twitch) scene became a big deal.
Not many were covering the PS1 though, until a few of the bigger channels started doing history and/or obscure gaming videos, which the Net Yaroze featured.
So, knowing when it launched, 1997 (UK/US), I thought to revisit it for the 20th anniversary and see if I can find some of the old members.
Learning and building - The Computer Science Craft
In the late 90’s I was a new C/C++ programmer but in 2016 after 15+ years coding in JAVA/SQL, I was very rusty and had to do a lot of relearning.
All the steps below, to me personally, are the kernel of how a programmer crafts robust & portable software!
I see it as a craft because of it’s complexity, intricacies and creative, adhoc nature.
Although, the Net Yaroze came out in 1997, the tools and skill sets haven’t been lost in time, it’s still the fundamentals of software today!
These build tools are core to most Operating Systems and it’s easy to replicate!
In fact, I have a separate game development project which builds for linux (32bit/64bit), windows(32bit), android(most SoC) all from the same code base.
I would say, I enjoy the C/C++ process over JAVA now, I find the more I learn about it, the more I want to learn more!
Always be learning and iterating on all aspects of your craft!
The development process
Starting - read a lot!
First, I read the user guide.
It explains everything.. some bits are vague, some are detailed.
Not having remembered any OpenGL, 3D maths and the hardware, I had to relearn everything from scratch.
I then reread the user guide, and it made more sense.
RIBIT: Read In Bed It’s Terrific! - A slogan I remember from my childhood.
Development environment - Getting Setup
Getting set up again wasn’t too bad, everything worked fine, in fact better!
Software like 2D/3D art tools and programming IDE’s are now a lot better then 20 years ago, cheaper or free!
The Sony toolchain was made in 1996 so it’s 16bit (DOS).
It works fine in DOSBOX, also the program used to send data to the Playstation (SIOCONS), also worked fine in DOSBOX(32bit only for me).
The development environment was a traditional UNIX setup, just replicated for DOS.
As a computer science student, we got taught on a SCO UNIX mainframe with monochrome (orange or green) dumb terminals.
I think that was very common for the time (80s to 90s) and a smart choice by Sony.
This environment is still widely used on most operating systems today, (dos, windows, linux, Mac, *nix, consoles, embedded devices, etc)!
Next, I got the basic demos working and experimented with them and started putting a few together.
Well, that was the easy bit!
GCC - GNU’s Compiler Collection
The Net Yaroze compiler was a GCC DOS port called DJGPP, and as to be expected, it’s ANSI C.
The compiler’s output format was ECOFF, which is the UNIX standard for Dynamically Linked Libraries (DLL).
When the Net Yaroze is loaded with the boot CD-ROM, it loads the actual library into RAM, which the ECOFF program links to.
I assume, this was implemented like this to save on serial communication loading, which can be problematic and slow.
LD - GNU’s Linker
Optional, but for the Net Yaroze you need to know and check that the program is loaded to the correct locations.
The compiler outputs a mapfile with a listing of addresses of functions and variables and necessary to watch the heap space:
GDB - GNU’s DeBugger
GDB is not a visual debugger like Visual Studio/Eclipse etc.
It’s a console/terminal text based program which displays the source code and variables via a commands.
Again, it has a learning curve but it’s pretty quick to get the basics.
The Net Yaroze’s implementation of GDB remote debugger works but it’s common for it to drop out.
I use printf outputs to find most of my bugs, GDB is my last resort and very glad it’s there!
MAKE - GNU MAKE
I was a few months into it and had a single file that was about 5 thousand lines.
I could see how I wanted to structurer the different files (game, menu, etc).
I have a healthy fear of MAKE, another language syntax to struggle with!
But I was more keen to split it so, I switch to MAKE.
MAKE, is a pain to setup correctly, but when it’s setup, it’s great!
I knew this at the time but what I didn’t realise was just how important it was.
A good MAKE file automates repetitive tasks, ie building, running and backing up.
It’s a lot smarter then just a batch file!
IDE - Integrated Development Environment
I started with code::blocks but I didn’t like the horrible unmodifiable right click menu!
I’ve been using eclipse since 2003, so I switched to eclipse CDT (C/C++).
Eclipse also has a great ‘file revision history’, which comes very handy.
I never had codewarrior’s C++ IDE in the 90’s.
It had playstation communication built in, so it has a visual debugger!
I did recently buy it to try it out, I didn’t like it, but to be fair, I didn’t give it much time, I struggled with user interface.
On top of the build environment is the actual coding.
The Playstation library (libps) included everything the Net Yaroze developer had access to in the Playstation.
This included most things the pro developers had, except for low level GPU hardware access, video playback and other minor things.
As described in the GCC section, the dynamic library(500kb) resided in the Playstation’s 2mb RAM.
So obviously, the smaller the library, the more RAM the developer can use!
The library, by today’s standards (OpenGL/DirectX) is very very! small and simple!
It’s basically this:
A few standard C functions: input/output, string, maths
Sound functions: loading, playing, stopping, change volume, pitch etc.
Graphic (2D & 3D) functions: loading, rendering, lights, VRAM copy, 2D lines/boxes, system font etc
The Playstation has no floating point hardware, which means the hardware only processes whole numbers.
The library supports floating numbers via software, which means the CPU converts the numbers to integer and then back again to floats.
And not a very optimal use of hardware!
The GPU hardware design is still very much like today, the GPU and it’s VRAM (1mb) is separated from the CPU and it’s system RAM (2mb)
So there is no direct access to VRAM, the program (CPU) has to instruct the GPU via a draw command list and this is draw each frame.
Emulation - Faster development
Without getting into the legality debate, I’ll just say, I have no problem emulating what I’m developing.
Obviously this wasn’t an option the 90’s, but it’s a lot faster then communicating via serial!
However, emulation isn’t perfect, I couldn’t find an emulator which replicated the serial communication for remote debugging.
And more importantly, performance isn’t 100% accurate, which makes it great for most things but hardware performance testing is still crucial.
The asset pipeline is still a huge issue today, but specialised 2D & 3D formats have largely been replaced by common ones.
C Programming language
I’ve heard internet people actually say that C programming is harder then C++ programming, sigh! :
C is a much smaller language and far less complicated then C++, the only hard thing about C are pointers, but it’s not that hard to understand.
Back in my day, C or Pascal was a common first programming language taught at universities.
Cheat sheet - Documentation
There a lot of tricks to be aware of with any hardware target, the Playstation for what it was, really wasn’t that bad!
But little things, would catch me out, ie coordinate systems, converters, etc.
So I made a word file, organize it into sections and create an automatic index at the start.
When I get stuck with something, and unique enough a quick search didn’t help, I would note it in the correct section with the solution.
I do this, because it will come up again and after writing it down, I know I wrote it!
And if in 10 or 20 years time, I want to revisit it, the notes will save me a bit of time!
Admittedly, I don’t tend to document my code much, unless it’s a complex algorithm etc.
Always be documenting!
Record videos/screenshots - See your progress easily
For a few years now, I’ve gotten into the habit of whatever I’ve been working on, before moving on or going to sleep, I would do a quick demo of what I have done.
No talking, No editing.. just click record, show it off, click stop and upload.
I also keep all my recordings/screnshots backed up.
It’s a great way to see the progression of any work.
Searching youtube or twitter is a lot easier then looking for a build and building it!
I do this for myself to look back on my work, it motivates me to continue.
And you never know who else is watching!
Just don’t name the videos your game’s name, you don’t want people seeing it if they’re searching for a finished game!
Progress is slow & organic - record it!
Back up - A cheap & important safety net
Lastly and most importantly, working in the computing field gives you an appreciation of a good backup system!
The stupidest thing is to lose important data, it just shouldn’t happen - but it does!
As computing professionals, we have a healthy level of pessimism!
However, for my small, solo project using CVS, github etc is overkill and it brings another level of complexity which I didn’t want to go into.
So, I created a simple batch file which zips up the entire project folder, including executables, images, sounds etc.
The backups are date stamped in the filename, saved to a local backup folder and my local file server, and periodically archived online and moved to 2 mirrored USB drives.
After doing each backup, I log the filename and put some notes in a text file called log.txt, which looks like this:
Yaroze-20160912_235921_backup.zip game init working Yaroze-20160913_191905_backup.zip CRAP wall hit collisions, changing to out of box check Walls fixed to out box check
Yaroze-20160914_194217_backup.zip Map using 4bit, not good two maps working
Yaroze-20160915_201126_backup.zip Better map
Yaroze-20160915_210519_backup.zip Reverting back to old tmds and collisions (inbox check)
Currently, each file is 10mb.
Occasionally I will knowingly or accidentally brake the project, it usually compiles but there’s a logic bug which I can’t fix quickly.
I then extract the last backup and do a comparison of the two folders using WINMERGE or MELD(Linux).
I can quickly see the difference between the two and can fix the issue.
On very few instances, I didn’t like the performance of a major change, but thanks to my backups and logs, I could revert back to before starting the change, ie above.
Don’t be stupid, backup and log changes!
The Community - The Net in Net Yaroze
I got my Net Yaroze welcome letter in March 1999, admittedly late but it still was a valuable experience for me nonetheless.
This is a part of my welcome letter from my CV portfolio, below my “User ID” was my assigned password and it was signed by Paul Holman (see below).
It was simple setup yet very functional and effective.
Before I joined, there was an issue that PAL members couldn’t access US/JP Sony’s and member’s sites/newsgroups and visa versa.
But by the time I joined this was fixed, and we could see each region’s sites and newsgroups.
I would guess this was initially done like this because it mirrored the professional developer networks, which would have been region based and locked.
In 1999, I remember the US members were mostly gone, posting only a few times that I recall (not sure if that was a access or connection issue though), the JP members were some what active, posting a few times a month (5-10).
The PAL members were very active, posting a few messages each week, if not a few in a day!
While the manuals from Net Yaroze kit wasn’t very detailed, especially the hardware and Sony’s library, the Net Yaroze “Private Network” (or just network for short) was overloaded with real world examples and felt never ending!
The Net Yaroze Network comprised of a few separate services:
Personal website - A space to share and show off
Sony’s repository - Updates, documentation and tools
Sony’s newsgroups - “Real time” communication with Sony staff and community members
Sony’s Net Yaroze repository - Updates, documentation and tools
Sony’s private Net Yaroze site offered a single place for updates and help.
The site hosted a lot of files which were not distributed in the Net Yaroze kit.
This included a lot of tutorials developed by Sony’s UK/US/JP Net Yaroze support teams & members, these included 2D, 3D, audio, text display, controller pads and assembly of low level CPU caching (scratchpad buffer).
Some where even playable but rough games, ie Clone, breakout
More detailed help about the toolchain (gcc, gdb, ld), peripherals, file formats and a FAQ.
Sony also had a ‘Technical Requirements Checklist for Net Yaroze Software’, extract:
The recommendations and requirements described in this document are a subset of those listed in the document ‘Technical Requirements Checklist for PlayStation® Software’, to which all published PlayStation® titles must conform.
Compliance with the requirements listed below is mandatory for Net Yaroze software which is to be considered for inclusion in published PlayStation® titles such as magazine coverdiscs. Compliance with the recommendations however is not obligatory, but the guidelines described will help you create software whose operation is intuitive and consistent with other PlayStation® titles.
The rules and recommendations contained in this document are not intended to stifle your creativity as a programmer, but to ensure that all published PlayStation® software meets users’ expectations of usability and quality.
There was no official publishing channels as part of the Net Yaroze, in the FAQ it was suggested to record your work on VHS as a demo real.
Only the UK Official Playstation Magazine published games on the demo disk, so I think this document was only meant for PAL regions.
There was also a small game demo section called “The Immaculate Collection”, which were some of the coolest games/demo’s!
6/5/99 Aaron Gandaa & Steven Sheehy; Technical Demo X2
6/5/99 David Johnston & Mike Goatly; Time Slip
1/3/99 Roland Bottcher; Rev Your Engine
5/2/99 James Pretorius; Opera of Destruction
21/1/99 Alex Herbert; Super Bub
21/1/99 Steve Collins; Car Demo
20/8/98 Chris Chadwick; Blitter Boy
3/9/98 Charles Chapman; Total Soccer Yaroze
3/9/98 Robert Swan; Adventure Game
23/8/98 Middlesex University; Motion capture
Personal website - A space to share and show off
On the same web site as the repository, Sony offered each member web hosting, If I remember correctly, it was 25mb.
This was plenty as most people only had dial up connections back then. Everything was low resolution, mostly text and with heavily compressed, small images like gifs.
our personal sites were only viewable by other members and were traditional Unix (Sun Ultra 5) Per-user web directories
ie, My login was mgarcia, and my Net Yaroze personal site was: http://netyaroze-europe.com/~mgarcia
We could share our code, Net Yaroze executables, home made tools (converters, etc), images, personal things and even random adult content :/
Uploading was also unix user based and via FTP.
The landing page of the member’s site was an listing of the “latest update”, newest update first.
Each member could fill out a small form and write about their update and it would appear on the list.
This was later used like a crude messaging service (like twitter), it wouldn’t be uncommon to see holiday well wishes.
Sony’s newsgroups - “Real time” communication with Sony staff and community members
“Usernet Newsgroups” was a popular server/client protocol which resembled a hybrid of emails within groups and threads, similar to what web forums are today.
I remember loading up my Mozilla’s Newsgroup reader and downloading all of the days new posts and replying were I could, it was the human side of development.
The hardware and software was very technical, there was a lot of demo’s and tutorials but none were for absolute beginners.
I remember reading about Net Yaroze in an Australian or UK Official PS Magazine Similar to this one but an earlier issue, they’ve never sugar coated it, and always said that it a certain level of C programming is required.
Maybe at times, they understated that, because although C programming isn’t the easiest thing in the world, it was 1000 times easier then programming any other console before it in assembly.
So, the Net Yaroze newsgroup had its share of people with well meaning ideas but lacking the technical expertise!
And in 1999 I was definitely one of them:
And the newsgroups was the only place to go to when anyone got stuck or needed to ask for clarification.
The Net Yaroze didn’t resolute in any famous millionaires like modern times, but there were a few games commercially published and it introduced a few people into the industry.
Net Yaroze might not have had an official publishing path, it did open doors: Black & White demo from Peter Molyneux, ECTS September 1999