iOS Note: Panoramic Hotspots

This is just a note on some of the solutions I've implemented for features on a project we are doing at my job.

So we have a panoramic viewer on the iPad. There is an image mapped to a sphere and a camera in the center of the sphere. You can rotate around and zoom in on things.

The designers wanted the functionality to click on specifically designated areas to trigger events. For instance, touch this painting and you get info on the piece.

In the legacy code I inherited for the project( it was already roughed out in prototype ), these triggers were detected via point-radius calculations. Here is an XYZ-position, here is it's radius, do BLAH when a touch hits in that radius. This works fine to some level of accuracy but they quickly found that they needed not only more accuracy but a more flexible way for other designers on the team to author these areas without reaching into the compiled code.

So they asked me, and I said to move the problem from model space to image space. Drop the positions and distances and just work with the pixels since that's what you are interacting with anyway.

My solution was basically to modify the rendering pipeline a bit, and to add a message queue.

For the pipeline, setup framebuffer rendering so that you aren't only writing to the screen but also to an offscreen buffer. Note that this will be a framebuffer not a renderbuffer. What do we draw to this offscreen buffer? We will have the artists/designers copy the panoramic texture and draw, in solid 1 color colors (no gradients or whatnot) the areas they would like to respond to touch. They will paint on their touch areas. With rgb 255-255-255 you get around 16,000,000 colors? That's 16 million unique touch areas. Plenty.

Now you can just glReadPixels a single pixel at the touch screen position from the offscreen bufer to get the color they touched.

What I do is, on a touch, I push a "request" for a sampling into queue-A. Queue-Ais the "request mailbox", the renderer's "inbox". The renderer checks this when it's submitting render calls and if it see's any requests in it's inbox it it calls glReadPixels and puts the result into queue-B, the "response mainbox", this is the renderer's "outgoing box". The rest of the application checks this outgoing box regularly to see if anything is in it. It will take any mail from it, which will be an rgb color, and will check a table it has to see what event trigger given the color. And that is it.

The users don't see this rainbow collision map because it's drawn offscreen. You don't have to transform any coordinates because all of that is done in the shaders. The transforms are the exact same as those performed on the screen image also so the collisions match up 1-1 regardless of any zoom or w/e features you do. The designers get per-pixel accuracy. It's a solid solution (has been so far at least ).

Caveats include not working in the simulator. I believe this is because the technique is hardware accelerated and thus dependent on. well. hardware and not just simulated architecture.

OpenGL explained: Framebuffer Object & Renderbuffer

Upon looking around for some info on an issue I'm currently having, I found the following posted by user "Nicol Bolas" on StackOverflow. I've read a few of this user's posts and they are quite knowledgeable on OpenGL related things.

Also, Nicol Bolas is the name of the chaos dragon from Magic The Gathering. Coincidence? Lol.

Anyway, the description is quite concise in my opinion.

A framebuffer object is a place to stick images so that you can render to them. Color buffers, depth buffers, etc all go into a framebuffer object.
A renderbuffer is like a texture, but with two important differences:

1. It is always 2D and has no mipmaps. So it's always exactly 1 image.

1. You cannot read from a renderbuffer. You can attach them to an FBO and render to them, but you can't sample from them with a texture access or something.

So you're talking about two mostly separate concepts. Renderbuffers do not have to be "for depth testing." That is a common use case for renderbuffers, because if you're rendering the colors to a texture, you usually don't care about the depth. You need a depth because you need depth testing for hidden-surface removal. But you don't need to sample from that depth. So instead of making a depth texture, you make a depth renderbuffer.
But renderbuffers can also use colors rather than depth formats. You just can't attach them as textures. You can still blit from/to them, and you can still read them back with glReadPixels. You just can't read from them in a shader.

http://stackoverflow.com/questions/9850803/glsl-renderbuffer-really-required

So you can't really use a renderbuffer for anything yourself. You put it in there so the system has a place to write things.

The framebuffers are for things you plan to work with.

D Programming: OpenGL 3.x+ (for Windon't 7) Part 3

First of all, here are some important links you need to bookmark and frequent for D/Derelict/MonoD/COOLSHIT:

• Derelict
• http://dblog.aldacron.net/page/2/
• Mono-D
• http://mono-d.alexanderbothe.com/
• Link to the original code (I modified it to fix a shader uniform error)
• http://www.digitalmars.com/d/archives/digitalmars/D/learn/Derelict3_running_35453.html

Things you'll need:

• zlib1.dll
• SDL2.dll
• SDL2_image.dll
• libpng15-15.dll

You will put these into your project next to your .exe file you run.

Here is the project code...

import std.stdio;      //required libpng15-15.dll + zlib1.dll placed next to .exe
import std.string; 
import std.conv; 
import std.path; 
import std.file; 
import derelict.sdl2.sdl; 
import derelict.sdl2.image; 
import derelict.opengl3.gl3; 

pragma(lib, "DerelictUtil.lib"); 
pragma(lib, "DerelictSDL2.lib"); 
pragma(lib, "DerelictGL3.lib"); 

SDL_Window *win; 
SDL_GLContext context; 
int w=800, h=600; 
bool running=true; 
uint shader = 0, vao = 0, tid = 0, colLoc = 0; 
int flags=SDL_WINDOW_OPENGL | SDL_WINDOW_BORDERLESS | SDL_WINDOW_SHOWN; 

bool initSDL_GL(){ 
   if(SDL_Init(SDL_INIT_VIDEO) < 0){ 
      writefln("Error initializing SDL"); 
      return false; 
   } 
   SDL_GL_SetAttribute(SDL_GL_CONTEXT_MAJOR_VERSION, 3); 
   SDL_GL_SetAttribute(SDL_GL_CONTEXT_MINOR_VERSION, 2); 
   SDL_GL_SetAttribute(SDL_GL_DOUBLEBUFFER, 1); 
   SDL_GL_SetAttribute(SDL_GL_DEPTH_SIZE, 24); 

   win=SDL_CreateWindow("3Doodle", SDL_WINDOWPOS_CENTERED, 
   SDL_WINDOWPOS_CENTERED, w, h, flags); 
    if(!win){ 
        writefln("Error creating SDL window"); 
      SDL_Quit(); 
      return false; 
     } 

    context=SDL_GL_CreateContext(win); 
    SDL_GL_SetSwapInterval(1); 
    
   glClearColor(0.0, 0.0, 0.0, 1.0); 
   glViewport(0, 0, w, h); 

    DerelictGL3.reload(); 

    return true; 
} 


bool initShaders(){ 
   const string vshader=" 
   #version 330 
   layout(location = 0) in vec3 pos; 
   layout(location = 1) in vec2 texCoords; 

   out vec2 coords; 

   void main(void) 
   { 
      coords=texCoords.st; 

       gl_Position = vec4(pos, 1.0); 
   } 
   "; 
   const string fshader="    
   #version 330 

   uniform sampler2D colMap; 

   in vec2 coords; 
   out vec4 fragColor;

   void main(void) 
   { 
      vec3 col=texture2D(colMap, coords.st).xyz; 

      fragColor = vec4(col, 1.0); 
   } 
   "; 

   shader=glCreateProgram(); 
   if(shader == 0){ 
      writeln("Error: GL did not assigh main shader program id"); 
      return false; 
   } 
   int vshad=glCreateShader(GL_VERTEX_SHADER); 
   const char *vptr=toStringz(vshader); 
   glShaderSource(vshad, 1, &vptr, null); 
   glCompileShader(vshad);    
   int status, len; 
   glGetShaderiv(vshad, GL_COMPILE_STATUS, &status); 
   if(status==GL_FALSE){ 
      glGetShaderiv(vshad, GL_INFO_LOG_LENGTH, &len); 
      char[] error=new char[len]; 
      glGetShaderInfoLog(vshad, len, null, cast(char*)error); 
      writeln(error); 
      return false; 
   } 
   int fshad=glCreateShader(GL_FRAGMENT_SHADER); 
   const char *fptr=toStringz(fshader); 
   glShaderSource(fshad, 1, &fptr, null); 
   glCompileShader(fshad);    
   glGetShaderiv(fshad, GL_COMPILE_STATUS, &status); 
   if(status==GL_FALSE){ 
      glGetShaderiv(fshad, GL_INFO_LOG_LENGTH, &len); 
      char[] error=new char[len]; 
      glGetShaderInfoLog(fshad, len, null, cast(char*)error); 
      writeln(error); 
      return false; 
   } 
   glAttachShader(shader, vshad); 
   glAttachShader(shader, fshad); 
   glLinkProgram(shader); 
   glGetShaderiv(shader, GL_LINK_STATUS, &status); 
   if(status==GL_FALSE){ 
      glGetShaderiv(shader, GL_INFO_LOG_LENGTH, &len); 
      char[] error=new char[len]; 
      glGetShaderInfoLog(shader, len, null, cast(char*)error); 
      writeln(error); 
      return false; 
   } 


   return true; 
} 

bool initVAO(){ 
   uint vbov, vboc; 
   const float[] v = [   -0.75f, -0.75f, 0.0f, 
                  0.75f, 0.75f, 0.0f, 
                  -0.75f, 0.75f, 0.0f]; 
   const float[] c = [   0.0f, 0.0f, 
                  1.0f, 1.0f, 
                  0.0f, 1.0f]; 
   glGenVertexArrays(1, &vao); 
   assert(vao > 0); 

   glBindVertexArray(vao); 

   glGenBuffers(1, &vbov); 
   assert(vbov > 0); 
   glBindBuffer(GL_ARRAY_BUFFER, vbov); 
   glBufferData(GL_ARRAY_BUFFER, v.length * GL_FLOAT.sizeof, v.ptr, GL_STATIC_DRAW); 
   glEnableVertexAttribArray(0); 
   glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, 0, null);          
   glBindBuffer(GL_ARRAY_BUFFER, 0); 

   glGenBuffers(1, &vboc); 
   assert(vboc > 0); 
   glBindBuffer(GL_ARRAY_BUFFER, vboc); 
   glBufferData(GL_ARRAY_BUFFER, c.length * GL_FLOAT.sizeof, c.ptr, GL_STATIC_DRAW); 
   glEnableVertexAttribArray(1); 
   glVertexAttribPointer(1, 2, GL_FLOAT, GL_FALSE, 0, null);          
   glBindBuffer(GL_ARRAY_BUFFER, 0); 

   glBindVertexArray(0);    

   return true; 
} 

bool initUniforms(){ 
      glUseProgram(shader); 
   colLoc=glGetUniformLocation(shader, "colMap"); 
   if(colLoc == -1){writeln("Error: main shader did not assign id to sampler2D colMap"); return false;} 

      glUseProgram(shader); 
      glUniform1i(colLoc, 0); 
      glUseProgram(0); 

   return true; 
} 

bool initTex(){ 
   assert(exists("4.png")); 
   SDL_Surface *s=IMG_Load("4.png"); 
   assert(s); 

   glPixelStorei(GL_UNPACK_ALIGNMENT, 4); 
   glGenTextures(1, &tid); 
   assert(tid > 0); 
   glBindTexture(GL_TEXTURE_2D, tid); 

   int mode = GL_RGB; 
   if(s.format.BytesPerPixel == 4) mode=GL_RGBA; 
    
   glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); 
   glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);    
   glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT); 
   glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT); 

   glTexImage2D(GL_TEXTURE_2D, 0, mode, s.w, s.h, 0, mode, GL_UNSIGNED_BYTE, flip(s).pixels); 

   SDL_FreeSurface(s); 
   return true; 
} 

//thanks to tito http://stackoverflow.com/questions/5862097/sdl-opengl-screenshot-is-black 
SDL_Surface* flip(SDL_Surface* sfc) 
{ 
     SDL_Surface* result = SDL_CreateRGBSurface(sfc.flags, sfc.w, sfc.h, 
         sfc.format.BytesPerPixel * 8, sfc.format.Rmask, sfc.format.Gmask, 
         sfc.format.Bmask, sfc.format.Amask); 
     ubyte* pixels = cast(ubyte*) sfc.pixels; 
     ubyte* rpixels = cast(ubyte*) result.pixels; 
     uint pitch = sfc.pitch; 
     uint pxlength = pitch*sfc.h; 
     assert(result != null); 

     for(uint line = 0; line < sfc.h; ++line) { 
         uint pos = line * pitch; 
         rpixels[pos..pos+pitch] = 
             pixels[(pxlength-pos)-pitch..pxlength-pos]; 
     } 

     return result; 
} 


int main(){ 
   try{ 
        DerelictSDL2.load(); 
    }catch(Exception e){ 
        writeln("Error loading SDL2 lib"); 
      return false; 
    } 
    try{ 
        DerelictGL3.load(); 
    }catch(Exception e){ 
        writeln("Error loading GL3 lib"); 
      return false; 
    } 
   try{ 
        DerelictSDL2Image.load(); 
    }catch(Exception e){ 
        writeln("Error loading SDL image lib ", e); 
      return false; 
    } 
    
   writeln("Init SDL_GL: ", initSDL_GL()); 
   writeln("Init shaders: ", initShaders()); 
   writeln("Init VAO: ", initVAO()); 
   writeln("Init uniforms: ", initUniforms()); 
   writeln("Init textures: ", initTex()); 
    
   while(running){ 
      SDL_Event e; 
      while(SDL_PollEvent(&e)){ 
         switch(e.type){ 
            case SDL_KEYDOWN: 
            running=false; 
            break; 
            default: 
            break; 
         } 
      } 
      glClear(GL_COLOR_BUFFER_BIT); 

      glUseProgram(shader);       

      glBindVertexArray(vao); 

      glActiveTexture(GL_TEXTURE0); 
      glBindTexture(GL_TEXTURE_2D, tid); 

      glDrawArrays(GL_TRIANGLES, 0, 6); 

      glBindTexture(GL_TEXTURE_2D, 0); 
      glBindVertexArray(0); 
      glUseProgram(0); 

      SDL_GL_SwapWindow(win); 
   } 

   SDL_GL_DeleteContext(context); 
   SDL_DestroyWindow(win); 
   SDL_Quit(); 

   return 0; 
}

So just paste that into your MonoD project's "main.d" and build/compile/run it. You can exit the program by pressing the "ESC" key. Hope this helps speed up your starting up.

D Programming: OpenGL 3.x+ (for Windon't 7) Part 2

To start we need to... start.

You will need to download the D installer, the Mono IDE, the MonoD extension, Derelict3, and some things required to get SDL2 to work.

I will be hosting everything on a single page so you don't have to crawl around for it. These downloads will become outdated but should (for the foreseeable future) work together just fine.

So if you want the latest/greatest make sure you DON'T use my downloads and actually go get copies of these things yourself.

Download everything from me.

1. https://code.google.com/p/brollace-blog-filehost/source/browse/d_pack.zip
2. On the right of the page is the link "view raw file".
3. Right click the link and "save as" to download the zip.

Setup a storage directory

1. Go to your "Documents" and create a folder called "DontMove".
2. In this folder create a folder called "D".
3. Unpack the contents of "d_pack.zip" to "DontMove\D" so that, for instance, you can observe the following path: "DontMove\D\4.png", and NOT "DontMove\D\d_pack\4.png".

Install D Programming Language:

Uses the d_pack files:

• dinstaller.exe

1. Run the "dinstaller.exe".
2. I put my installation on "C:\D\<my stuff installed here yo>"

Setup Derelict3 for OpenGL 3.X+

Now this part was painful so please do allow me to share the fruits of my labor with you to end the chain of suffering.

Uses the d_pack files:

• aldacron-Derelict3-b4f810c.zip

1. Unzip the aldracon package.
2. Go to "import" so you see a folder containing "derelict".
3. Copy the folder "derelict" with all the stuff in it and paste it into (for example on my machine) "C:\D\dmd2\src".
4. The folder should look something like "derelict, dmd, druntime, phobos".
5. Go to "aldracon/lib" and copy everything to (for example on my machine) "C:\D\dmd2\windows\lib".
6. You should have just copied a bunch of .libs into a folder of other .libs.
7. Open up "C:\D\dmd2\windows\bin\sc.ini" in some editor like Notepad++ or Scite or ConTEXT.
8. Change the line: [1] to look something like [2]...

[1] DFLAGS="-I%@P%\..\..\src\phobos" "-I%@P%\..\..\src\druntime\import"

[2]DFLAGS="-I%@P%\..\..\src\phobos" "-I%@P%\..\..\src" "-I%@P%\..\..\src\druntime\import"

Install MonoD

Uses the d_pack files:

• MonoDevelop-3.0.4.7.msi
• MonoDevelop.D_0.4.1.4_MD3.0.4.7.mpack
• ResourceCompiler.zip

1. http://mono-d.alexanderbothe.com/?page_id=9

.

.

.

.

You should now be able to create D programs in MonoDevelop using MonoD.

Next I will go over how to write your first D OpenGL3.X+ app using SDL2.

D Programming: OpenGL 3.x+ (for Windon't 7) Part 1

D. Just say it. D...

Mmm....

I really want to get away from C++ (lol impossible ikr).  So I'm going to invest my time and projects into developing with D.

These posts will be tutorials on how to start up D programming, and then on how to write an OpenGL 3.X+ application in D.

A Good Place to Start OpenGL

Many people want to learn OpenGL. Most of these people have no idea where to start. About everyone will learn OpenGL in some deprecated or round-a-bout way that discourages them from continuing their studies.

Well that's stupid and I'd like to at least point people in the right direction for getting started. This definitely isn't a tutorial, but I hope that it does serve to broaden your knowledge of available OpenGL tools, libraries and resources.

Here are the resources:

• Clean Example Sample Projects: http://www.g-truc.net/project-0026.html#menu
• A means to make windows, get time, etc:  http://www.glfw.org/
• A way to initialize extensions:  http://glew.sourceforge.net/
• Math for everything you will do:  http://www.g-truc.net/project-0016.html#menu
• To build the examples:  http://www.cmake.org/

Basically you should walk through the samples provided over at g-truc to get an idea of what it means to create a current generation OpenGL project. The authors say that the code is not for beginners but they are only being modest and probably just don't intend for them to be used by beginners. I can honestly say that they are some of the best examples of OpenGL code available.

To build them on Windows for Visual Studio 2010 you need to install cmake. After that open up command line by running it as administrator. In this cmd window change to the samples folder that contains the /data, /external, /samples... the top level of the folder. Run the command [cmake -G "Visual Studio 10"]. Do not use the cmake GUI for this as it won't work. Now you should have vs2010 projects built that you can explore by opening the .sln.

What I suggest that you do is copy the projects and get accustomed to how they use OpenGL to express themselves. Their image loading tool, GLI, is for bringing in images to the program to use as textures. FreeGLUT is a "make windows and other stuff" utility that some people use. I honestly think all GLUT stuff is lame and suggest you learn to use GLFW instead. GLM is a great way to encapsulate all of your math needs for things such as vectors and matricies (which you will make heavy use of). It gives you a portable math solution that works great with OpenGL.

3D Bezier Rails

If you'd like to have a 3D "rail" in your application you can use Bezier curves.

Rails are really useful for defining paths for objects to travel across.

Bay-Z-Ay curves can be really simple too for implementing rails. All you need are:

• A list of 3D points in the order of your path.
• A function to evaluate the position along this path from time 0 to 1.

Here is an example of a function one could use to get this bezier curve position (written in javascript but it doesn't really matter):

function atT( t: float,
     p0 : Vector3, p1 : Vector3, p2 : Vector3 ) : Vector3 {
        var t2 : float = t * t;
        
        var one_min_t : float = 1 - t;
        var one_min_t_2 : float = one_min_t * one_min_t;
        
        var x : float = one_min_t_2 * p0.x + 
         2 * one_min_t * t * p1.x +
         t2 * p2.x;
        var y : float = one_min_t_2 * p0.y + 
         2 * one_min_t * t * p1.y +
         t2 * p2.y;
        var z : float = one_min_t_2 * p0.z + 
         2 * one_min_t * t * p1.z +
         t2 * p2.z;
        
        return(Vector3(x,y,z));
    }

So given your list of points ( 0, 1, 2, 3, 4, 5, 6... ), you'd break them up into groups of 3 like ( [0,1,2], [2,3,4], [4,5,6]... ). Keep track of which group/segment your on and just interpolate from 0 to 1 to move along it. When you hit 1, increment to the next group/segment and restart at 0.