| To post the Grid on other sites, save this swf file Stand-alone clients: Windows Viewer Mac Viewer |
Main Grid
4682 pictures
If the Grid runs slowly, try this
| Directory: The display The server Origins The future Get involved Comment |
About the Grid
Zareste's Grid is a giant 2d board where anyone can add a picture, free of registration. It's designed so you can scroll around and view pictures using an out-of-the-way interface. Everything is grouped by space instead of categories, so you can wander around instead of searching for something specific.
How it works
A Grid can be huge - they have a height and width up to 32,768 pixels and thousands of pictures each. When you connect, you're downloading coordinates for every picture. That's a lot of coordinates, and in the old days of the Grid it caused a lot of bloat. These days the coordinates are only two bytes per image - one byte for the horizontal and one for the vertical. The system uses an actual grid of 128 pixels so it doesn't need to position each picture precisely. It reads the coordinates, and if the first two bytes are 2 and 3, the Grid sets the image at 256x384, if the second two bytes are 1 and 4, the image is put at 128x512. It then assumes that the first two coordinates in the file refer to 0.jpg, and the second two refer to 1.jpg, etc, so no file names have to be communicated.
This way, the picture can be set at positions up to 32,768x32,768 using only two bytes. So a coordinate file for 10,000 pictures will be 20kb, which is smaller than an all-text web page.
The data sends with little latency because the Grid uses flat files in my own database format instead of MySQL, taking out the 'middle man' from server requests.
The display
On a Grid with 5,000 pictures, you can't download them all at once. A lot of computers would explode if I did that. So it only displays pictures in the surrounding space - the applet size x 2. When you reach the edge of the currently displayed pictures, the references are dumped and a new display area is created. Your computer reads all the (thousands of) coordinates, figures out which pictures are in your display area, and then they appear from the middle outward (distances are calculated by the Pythagorean theorem 'X squared + Y squared = the square of the distance'). All of this happens in a few milliseconds while you scroll. A cache of 50 image previews is kept so they'll appear later.
The backmap uses parallax scrolling to move - just like those backgrounds in SNES games. The squares are scaled to .4 times the image sizes and move at .4 times their speed.
The overhead map (pressing Enter) is a flat bitmap calculated from the image coordinates when the applet starts. No remote files used.
Flick scrolling - A minor feature, but the Grid 'flicks' smoother than some programs. The difference is that the Grid records the mouse position from the previous frame instead of the previous mouse movement, and uses that as a reference for scrolling.
The server
A PHP program receives sent pictures and creates previews. Data on the pictures is kept in my own database format (not SQL) for speed and control. When you connect, you receive three streams:
-An image-position file
-A deletion list
-User info
The deletion list tells which pictures not to download. When I personally delete an image, the image-files are deleted but its positions are kept so not to disrupt the byte order in the position file. Each deleted image is represented by two bytes giving the image number. On deletion, the checksum hash of the image is also kept to prevent it from being reposted.
User info tells the browser name and applet location for in case the Grid needs to adjust for a browser or domain.
When you send an image, it streams to the server along with the position, and processed by a PHP file. The PHP creates a checksum hash of the image (a set of numbers that are unique to the file) and checks the hashes of other files to make sure the image is not a duplicate. The code then optimizes the image's filename (removing server-unfriendly characters and keeping the length under 32 characters), it creates a preview of the image, then saves the information on position, hash and filename. It returns the file information to the Flash, which displays the image.
The Grid client updates the board and chat box about every 10 seconds. The server keeps an update file showing the number of recent users, recent chat information, and recently sent pictures - all with timestamps. The Flash client retains a timestamp of the last time it updated, and any information with a more recent timestamp is relayed to the Flash. Once updates are received, the Flash updates the board, the chat and the user number. If no updates are received, it increases the update latency to 20 seconds, then 30 seconds, up to 60 seconds. This prevents unnecessary server relays.
Origins
The Grid is a simple concept that came easy. I was surfing Google Earth and thought "what if people could add anything to this?" Like a giant free-for-all board. I had to refine the idea a lot before it became practical.
My original plan was a huge environment where you could draw or model anything alongside other people, but programming difficulties and web limitations got in the way. So planned a rough proof-of-concept.
At first I wanted pictures full-size and downloaded in chunks like on Google Earth so you can zoom in and out, but this was very complicated and could cause re-compression artifacts in pictures, and users would constantly be zooming. So instead I used today's format with preview images. When a user clicks, the full-size image would come up, and it would move along with the mouse, similar to the zoom option in OS X, making it easy to view big pictures on any screen size.
The first working Grid was very rough. It had bugs everywhere, no maps, no chat box. It didn't have the 128-pixel 'grid' so people could put pictures anywhere, but they overlapped rampantly due to sync errors and bugs. The coordinate file (downloaded on connection) was much bigger. It was human-readable, causing its file-size to reach about 500KB.
BUT it was a hit. Flash sites and 4chan had a lot of fun with the new concept. So, after a few months of minor bug fixes, I started a major upgrade. The coordinate file was overhauled to today's denser machine-format, maps were added, the chat box was added, the files were converted to preserve pictures, and I made an swf container so viewers on other sites would always get the latest version.
Unfortunately the old Grid swf was still floating around out there, and it wouldn't support the new data format, and I couldn't fix it on other sites, so I left the old files on the server so these users could still access the Grid. After a few months, I cut off support for the old Grid.
The future
My big plan is to re-define the web using the Grid, and have all content on a giant board for 'free-form' viewing so sites won't be separated as they are today. Search engines will be less important because you can move around and view things grouped by space, though tags can be placed for searching. Domains will be unnecessary, and all servers can be tracked similar to today's DNS system but without domain names. You'll see text and pictures and sounds and videos and games laid out wherever people want.
Hosting will be different. Servers will host blocks of the overall Grid, giving them control over what goes on in those blocks. If you own a block and you don't want anyone altering certain areas, you can section off those areas, or restrict users to text-only, or list who can alter it. Software will use persistent sockets to elliminate delays between downloads (a problem in today's web).
P2P swarming will keep everything at top speed with less server usage. It will let ordinary users to host blocks - maybe even dial-up users. It will also keep blocks alive after the server goes down.
The Grid's client programs will be browsers and designers at the same time. So you can draw a picture on the Grid, add some text, and set permissions. All content will be under a creative commons license with derivative works allowed, no exceptions. Pro-copyrighters might fight me on that, but the system is arranged so they can't beat it.
All of this can be done on today's hardware.
Get involved
We could always use:
- Programmers (Actionscript, PHP, Java, Unix, Win/Linux/Mac languages, P2P networking)
- Mirror hosts
- Anyone who just wants to help
Programmers: nearly any language is welcome. I'd like to make clients on other platforms like Java or as executables for Windows/Linux/Mac, because these can use P2P networks and relieve bandwidth, and run faster than the current stand-alone executables. Unix socket programming is useful for making dynamic servers, speeding up response times and relieving CPU.
Mirrors: If you have a web host (preferably dedicated) that allows hosting for other sites, I can setup a PHP relay so some of the user-sent pictures can be securely sent to your server and hosted.
If you have any other skills that could help, leave me a note. I'm not picky.
To offer any help, leave a note below with your contant information (I can delete the post if you want to avoid spam bots)
blog comments powered by Disqus