I'm surprised the agency that administered the test did not provide you with study materials in advance of the exam. Seems quite unfair to be tested on something you've never seen before.
Depends on the test, Norm. If the test is considered a general test in nature, where anyone should be able to answer the questions, then there is no study guide or materials that would need to be provided. You would see this with depts that ask for basic requirements for the job, like 18, HS or GED, and driver's license. Whereas if the test is focused on specific areas of FF, then there would/should have a study guide. The other aspect is if there is a dept specific portion of the test, that information could be provided before that portion of the test is given.
Essentially, a study guide or materials does not necessarily have to be provided in advance of an exam. The only way it would be "unfair" is if the test focuses upon dept specific questions, SOGs, operations, etc and that information isn't provided before hand. Yet, even if it did, that info could be provided right before the questions. For example, the written test for the City of Milwaukee was a general test, but, the first 10 to 15 questions pertained to information that was dept specific. That information was provided by a proctor reading some information that you had to remember, such as helmet front color for a truck and engine company, size of water supply pipeline, etc. It would only be "unfair" if there was no information provided at all.
To the OP, never seen such a test. I have been through the BPAD testing, which is video based with conflict type of resolution. You sat in a room with a video monitor and a camera recording you. You had to interact with the actors in the video and respond as you were right there. I found it to be a wierd situation, but did well on it. One can look up BPAD and look at suggestions and ways to study etc from that. Never seen or heard of the other aspects until now. Which dept was doing this testing....if you don't mind?
I took 2 of the very same test you are describing in North Carolina. I found it actually a fairly unique testing process compared to what NY does. I think the video based tests give you an evaluation of an individuals natural ability to interact with people and your natural ability to problem solve. For an entry level exam I think it is very fair. It seems that if you have a natural ability to perform well in these situations then the "lifestyle" of being a firefighter is relatively easy. I am not sure these are qualities that you can train or educate someone to posses.