I already asked this on reddit, with almost no responses. I hope here is the right place to be asking these kind of questions. So, here we go.
I have developed a technique in some diagrammatic notation that should increase the understandability of the diagrams.
Now in order to empirically prove that this technique significantly increases the understandability of the diagrams, I need to conduct an experiment.
I plan to group participants into two groups, G1 which will use my technique and G2 which won’t have my technique available. Both groups will receive a diagram and they will answer a series of questions. I hypothesize that G1 will have significantly better results than G2, since they will be using my technique.
Of course, I was taking into the consideration the complexity of the diagram as well – maybe the complexity of the diagram can influence the results, e.g. in a case of a diagram with low complexity my technique might not be significantly better than plain diagram. So, both groups will receive two diagrams, one complex and one simple. The order in which they will receive this diagram will be randomized, to ensure that the sequence in which they obtain the diagrams won’t have an effect on the results.
Now I’m trying to do a research model.
I have two independent variables: the approach (my technique or plain diagram) and diagram complexity (low and high). Also, I have several dependent variables (number of correct answers, efficiency, etc.).
At first I thought this is a 2×2 factorial design, something like this:
But then I kept thinking… If I’d like to have 2×2 design:
- first group would be with my technique and low complexity diagram
- second group would be with my technique and high complexity diagram
- third group would be without my technique and low complexity diagram
- forth group would be without my technique and high complexity diagram
but this is not the case, or rather, this is not what I want. I don’t want to have 4 groups, but 2: the ones that use my technique (G1) and the ones that don’t (G2), both solving high and low complexity diagrams.
So I was wondering what is the diagram complexity in my case? Considering the above, I know it’s not an independent variable, because I don’t have a 2×2 factorial design.
I read some more about research methods, and I came across “moderating variable”. So, the diagram complexity might influence the relationship between the approach and dependent variables. something like this:
Such model could be analyzed using Analysis of Covariance (ANCOVA). However, in my case, the moderating variable has two values for each participant, low and high.
So my questions are as follows:
- am going in the right direction? Is the diagram complexity moderating variable or is my case best suited for 2×2 factorial design?
- is it a problem if you have a moderating variable that has 2 values and they are applied for each participant?
Thanks in advance, any kind of feedback is greatly appreciated.