Advanced programming problem
by Michael Perry · in Technical Issues · 10/02/2007 (12:08 pm) · 10 replies
*** WARNING: THIS THREAD IS NOT FOR THE WEAK OF HEART or PROGRAMMING NEWBIES ***
Hey all. I've posted in the general programming forum because this is mostly unrelated to Torque development. I am attempting to emulate mouse/keyboard/joystick messages which are generated based on data polled from a Wii remote/nunchuck.
The application I'm writing is meant to emulate input devices/messages WITHOUT having access to a game's source code. That's right, I'm adding wii remote support to ALL PC games. Implementing Wii remote functionality in Torque took less than a day. This app is much more challenging.
The nunchuck will control typical arrow or WASD movement, where as the Wiiremote will act as the mouse pointer and mouse buttons.
The code related to the Wii device is complete, so I'm down to the Win API stuff involved in input emulation. Up until now it's been "one step forward, two steps back." I've finally broken some ground, though, and I was wondering if someone could help me finish it off.
Here's what works:
Hey all. I've posted in the general programming forum because this is mostly unrelated to Torque development. I am attempting to emulate mouse/keyboard/joystick messages which are generated based on data polled from a Wii remote/nunchuck.
The application I'm writing is meant to emulate input devices/messages WITHOUT having access to a game's source code. That's right, I'm adding wii remote support to ALL PC games. Implementing Wii remote functionality in Torque took less than a day. This app is much more challenging.
The nunchuck will control typical arrow or WASD movement, where as the Wiiremote will act as the mouse pointer and mouse buttons.
The code related to the Wii device is complete, so I'm down to the Win API stuff involved in input emulation. Up until now it's been "one step forward, two steps back." I've finally broken some ground, though, and I was wondering if someone could help me finish it off.
Here's what works:
- Mouse movement emulation[li]Mouse button press/release emulation[li]Mouse wheel scroll emulation[li]Keyboard press/release emulation for 95% of the games I've tested
- Emulating joystick buttons and maybe axis (let's start with buttons)[li]Solve problem with Operation Flashpoint (which uses DirectInput) not taking emulated keyboard input
- To emulate the keyboard, I had to drop down to a low level input system since DirectInput doesn't like to share keyboard data[li]Despite Op. Flashpoint using DirectInput, nothing I've tried so far can emulate the keyboard (keybd_event, ::SendInput, SetWindowsHookEx)[li]I have no clue on how to emulate the joystick using ::sendInput. I have read through forums and the msdn, but I just don't know how to fill out the hardware input structure (msg, wparam, lparam)
- I am writing this in C++[li]I am using Visual Studio 2005[li]I am emulating all of the mouse commands using ::SendInputI am emulating the keyboard input with the SetWindowsHookEx method[li]The app must run without focus[li]This app is currently running on Windows API (sorry Linux/MAC people, I have to get it working before I port it)
About the author
Programmer.
#3
i'll upload the binary to my site and link it in a bit. with the core app ready, i'm writing an mfc dialog which has 3 functions...
1. it will let the user pick the game they want to launch
2. the user can load in a script file which maps controls for the above game to the wii remote and nunchuck. the dialog not only loads scripts, it also has edit controls and listboxes to modify and save your control scheme out for later use (kind of like joystick profiles or TGE actionmaps). glovepie is a big inspiration for my app.
3. launches the base app which drives the wii controller.
when the user hits launch, the wii app will execute, assign the proper emulation controls, and launch your game/app.
i'll post in a bit with the link. as it stands right now, the controls are hard coded, so you can't change the mapped keys yet, but i'll fix that tomorrow. be sure to read the README.txt. you will need a bluetooth adapter and a bluetooth stack which replaces the default windows stack. i use bluesoleil
also, to get IR motion data, you'll want two LED lights (wii sensor bar or other). hell, even 2 candles will work
hey, i didn't make the hardware rules...i'm just a programmer who likes to create wacky applications.
post in a bit (on a different computer, because this keyboard is driving me insane)
10/04/2007 (4:22 pm)
Forgive the bad grammar, lack of capilization, and vspelling verrors. my girlfriend spilled milk on my keyboard and all the keys are sticky, or won't work at all.i'll upload the binary to my site and link it in a bit. with the core app ready, i'm writing an mfc dialog which has 3 functions...
1. it will let the user pick the game they want to launch
2. the user can load in a script file which maps controls for the above game to the wii remote and nunchuck. the dialog not only loads scripts, it also has edit controls and listboxes to modify and save your control scheme out for later use (kind of like joystick profiles or TGE actionmaps). glovepie is a big inspiration for my app.
3. launches the base app which drives the wii controller.
when the user hits launch, the wii app will execute, assign the proper emulation controls, and launch your game/app.
i'll post in a bit with the link. as it stands right now, the controls are hard coded, so you can't change the mapped keys yet, but i'll fix that tomorrow. be sure to read the README.txt. you will need a bluetooth adapter and a bluetooth stack which replaces the default windows stack. i use bluesoleil
also, to get IR motion data, you'll want two LED lights (wii sensor bar or other). hell, even 2 candles will work
hey, i didn't make the hardware rules...i'm just a programmer who likes to create wacky applications.
post in a bit (on a different computer, because this keyboard is driving me insane)
#5
@Aun - To answer honestly, it performs a similar function as Glovepie, but it is in no way close to being as polished and feature-heavy as Glovepie.
As I said, Glovepie is my inspiration. I knew what I wanted, so I started researching and programming. When I hit my last major obstacle, I had a one-on-one with Carl Kenner (author of Glovepie) via e-mail and he helped me troubleshoot my problem.
Change of plans though. Hmm, without revealing details as to why, if you want to see the app at work send me an e-mail at mperry@zombieshortbus.com.
This works a little better than posting the app publicly.
Thanks for the interest, and I look forward to your feedback.
10/04/2007 (6:54 pm)
Ok. Back in gear with a working keyboard =)@Aun - To answer honestly, it performs a similar function as Glovepie, but it is in no way close to being as polished and feature-heavy as Glovepie.
As I said, Glovepie is my inspiration. I knew what I wanted, so I started researching and programming. When I hit my last major obstacle, I had a one-on-one with Carl Kenner (author of Glovepie) via e-mail and he helped me troubleshoot my problem.
Change of plans though. Hmm, without revealing details as to why, if you want to see the app at work send me an e-mail at mperry@zombieshortbus.com.
This works a little better than posting the app publicly.
Thanks for the interest, and I look forward to your feedback.
#6
Thanks !
Aun.
10/04/2007 (7:04 pm)
@ Micheal , Don't get me wrong , I'm not trying to compare your application with Glovepie or anything. I'm sure every programmer have their needs to find some excitement with some fun and new apps. But right now I'm also using Glovepie for a prototype and just wanted to know if your app have any new features that might sound useful.Thanks !
Aun.
#7
I do have plans for making it shine, and there are some games where my app worked and Glovepie didn't. There are bugs, there are flaws, and there are features to be implemented.
So, this is as good of a time as any to ask what features you would like to see in this app. What do you want to see or have at your disposal?
*Edits* - Grammar improved and repetition removed....damn alcohol....
10/04/2007 (7:14 pm)
@Aun - Thanks for the response. I've only been working on this for 3 days now, so there isn't much "glamour" or gold plating.I do have plans for making it shine, and there are some games where my app worked and Glovepie didn't. There are bugs, there are flaws, and there are features to be implemented.
So, this is as good of a time as any to ask what features you would like to see in this app. What do you want to see or have at your disposal?
*Edits* - Grammar improved and repetition removed....damn alcohol....
#8
gguser
redsoxrule
Very early beta version. Didn't get to update it today, or add the GUI...got side tracked by life. Speaking of which, did you guys know there is an "outside?" Fresh air, amazing rendering, HDRL, and amazing interaction capabilities. The AI for the people is a little dumbed down, though. =)
10/05/2007 (9:11 pm)
GG Stuffgguser
redsoxrule
Very early beta version. Didn't get to update it today, or add the GUI...got side tracked by life. Speaking of which, did you guys know there is an "outside?" Fresh air, amazing rendering, HDRL, and amazing interaction capabilities. The AI for the people is a little dumbed down, though. =)
#9
10/05/2007 (9:15 pm)
I've found that if you use your "Pick Pocket" skill on large groups of mobs, you can sometimes score a "wallet" that contains pieces of paper that can then be used to make the people AI a little more responsive. YMMV.
#10
Ooook. Back to some serious work. So, aside from making a GUI to make this app more user-friendly, I have one last little bug to take care of: varied mouse sensitivity.
When using the Wii remote, I emulate the mouse movement based on the IR data I parse, using SendInput.
In Windows and GUI screens, the movement is spot on. However, once I get into the game itself (any game really), the sensitivity gets blown way out of proportion. I believe this is most likely due to the way DirectX games handle buffered mouse input.
So what I need to figure out is a new algorithm for emulating the mouse movement. Otherwise, this app will only work if you greatly decrease the mouse sensitivity to a low decimal number (like 0.005).
Anyone have any incite? I know IGC is going on right now, so no big rush.
10/10/2007 (7:04 am)
Stephen - Funny stuff =)Ooook. Back to some serious work. So, aside from making a GUI to make this app more user-friendly, I have one last little bug to take care of: varied mouse sensitivity.
When using the Wii remote, I emulate the mouse movement based on the IR data I parse, using SendInput.
In Windows and GUI screens, the movement is spot on. However, once I get into the game itself (any game really), the sensitivity gets blown way out of proportion. I believe this is most likely due to the way DirectX games handle buffered mouse input.
So what I need to figure out is a new algorithm for emulating the mouse movement. Otherwise, this app will only work if you greatly decrease the mouse sensitivity to a low decimal number (like 0.005).
Anyone have any incite? I know IGC is going on right now, so no big rush.
Employee Michael Perry
ZombieShortbus
AH HA! I figured it out! I now have an application, which runs in the background, that emulates mouse and keyboard messages using the Wiimote and the nunchuck/classic controller!
I'll be polishing it over the next few days, so if anyone wants the executable to play around with, just let me know.
(Much thanks to Carl Kenner, creator of Glovepie, for his help)