**Basic
Concepts of Quantitative Research**

Dr. R. Ouyang

**Method**

** **

*Selection of a sample*

Population and sample: So far we have talked about introduction, literature review and methods in a research. Next we will discuss the subject. Subjects mean the population or samples researcher used for study. Population is the group of interest to the researcher, the group to which she or he would like to the results of the study to be generalizable. The population that the researcher would ideally like to generalize to is referred to as the target population; the population that the researcher can realistically select from is referred to as the accessible or available population.

If the group of interest is unmanageably large or geographically scattered, study of this group could result in considerable expenditure of time money and effort. Therefore, selecting a sample is needed and is a very important step in conducting a research study. A “good” sample is one that is representative of the population from which it was selected.

Regardless of the specific techniques used for sampling, the steps in sampling are essentially the same: Identification of the population, Determination of the required sample size, and Selection of the sample. There are four basic sampling techniques: random sampling, stratified sampling, cluster sampling, and systematic sampling.

** FOUR sampling techniques are: **Random
sampling, Stratified sampling, cluster sampling, and systematic sampling.

**Random sampling** is the process of selecting a sample
in such a way that all individuals in the defined population have an equal and
independent chance of being selected for the sample. It is considered as the best single way to obtain a representative
sample that is required by inferential statistics.

Steps of random sampling:

1. Identify and define the population;

2. Determine the desired sample size;

3. List all members of the population;

4. Assign all individuals on the list a consecutive number from zero to the required number, for example, 000 to 249, 00 to 99;

5. Select an arbitrary number in the table of random numbers (close eyes and point);

6. For the selected number look at only the appropriate number of digits (2 or 3 digits);

7. Match the selected number with the number assigned to individuals in the population, if the number corresponds to the number of assigned number of individual, the individual is in the sample;

8. Go to the next number in the column and repeat step 7;

9. Repeat step 8 until the desired number of individuals has been selected for the sample.

Example-- Label 5000 teacher with the number 0000 - 4999, then randomly point a number in the random number table, match the last four-digit number of the number in the table with the teacher’s label number. If it is matched the teacher is selected in the sample. Repeat the process until 500 teachers are selected as the required sample size.

**Stratified sampling** is the process of selecting a
sample in such a way that identifies subgroups in the population are
represented in the sample in the same proportion that they exist in the
population. In this process, random
sampling is done more than once; it is done form each subgroup.

Steps of stratified sampling:

1. Identify and define the population;

2. Determine desired sample size;

3. Identify the variable and subgroups for each the researcher wants to guarantee appropriate representation;

4. Classify all members of the population as members of one of the identified subgroups;

5. Randomly select an “appropriate” number of individuals from each of the subgroups, “appropriate” meaning either a proportional number of individuals or an equal number of individuals.

Example: Suppose one is interested in comparing the performance of students of different IQ levels, following two different methods of mathematics instruction. One can have 300 eighth graders classified in to high (>115), average (85-115), and low (< 85) IQ group, randomly select 30 students from each group, and then randomly assign 15 from each 30 students to experimental or control groups for the study.

**Cluster sampling** is sampling in which groups, not
individuals, are randomly selected. All
the members of selected groups have similar characteristics.

Steps of cluster sampling:

1. Identify and define the population.

2. Determine the desired sample size.

3. Identify and define a logical cluster.

4. List all clusters that comprise the population.

5. Estimate the average number of population members per cluster.

6. Determine the number of clusters needed by dividing the sample size by the estimated size of a cluster.

7. Randomly select the needed number of cluster (using a table of random numbers).

8. Include in the study all population members in each selected cluster.

Example: One needs to have 500 teachers from 5000 for the study. There are 100 schools with 50 teacher in each. In cluster sampling, one randomly select 10 schools for study and have 500 teachers in the sample. The problem of this sampling is that inferential statistics are not appropriate for analyzing data resulting from a study using cluster sampling. Inferential statistics generally require random sampling.

**System sampling** is sampling in which individuals are
selected from a list by taking every K^{th} name. If the list of the population is randomly
ordered, this sample can be considered as a random sample

Steps of systematic sampling:

1. Identify and define the population.

2. Determine the desired sample size.

3. Obtain a list of the population.

4. Determine what k is equal to by dividing the size of the population by the desired sample size.

5. Start at some random place at the top of the population list.

6. Starting at that point, take every Kith name on the list until the desired sample size is reached.

7. If the end of the list is reached before the desired sample is reached, go back to the top of the list.

**Sample size**:
Sample should be as large as possible.
Minimum acceptable sample size depends on the type of research. For causal-comparative, correlational
research 30 in each group, and 15 in experimental research are generally
recommended as minimum sample size.

Nonprobability sampling that includes:

1) convenience sampling: the use of volunteers and existing groups

2) judgment sampling: the use of believed groups

3) quota sampling: the use of quota in the situation of not all members of the population be possible listed for sampling.

**Selection of measuring instruments**

All research studies involve data collection. There are three major ways to collect dada: 1) administrate a standardized instrument; 2) administrate a self-developed instrument; 3) record naturally available data.

*Validity and Reliability*

**Validity** means the degree to which a test measures
that it is supposed to measure and, consequently, permits appropriate
interpretation of scores. The concept
of validity is that the validity for whom or for what. Since tests are designed for a variety of
purposes, and since validity can be evaluated only in terms of purpose, the
validity is categorized in several different types: Content, construct,
concurrent and predictive validates.

**Content
validity**: Content validity is the
degree to which a test measures an intended content area. It requires both item validity and sampling
validity. Item validity is concerned
with whether the test items represent measurement in the intended content area,
and sampling validity is concerned with how well the test samples the total
content area. The content validity is
determined by expert judgment.

**Construct
validity**: Construct validity is the
degree to which a test measures an
intended hypothetical construct. It is
not easy task to process a test of construct validity. Generally, a number of independent studies
are required to establish the credibility of a test of a construct.

** Concurrent
validity**: It is the degree to which
the scores on a test are related to the scores on another, already established,
test administered at the same time, or to some other valid criterion
available at the same time. The concurrent validity is determined by the
resulting number, validity coefficient.
If the coefficient is high, the test has good concurrent validity.

** Predictive
validity**: Predictive validity is
the degree to which a test can predict how well an individual will do in a
future situation. The predictive
validity of a test is determined by establishing the relationship between
scores on the test and some measure of success in the situation of
interest. It is determined by the
resulting number, validity coefficient.
If the coefficient is high, the predictive validity is good. Generally, 0.5 might be acceptable
predictive validity in certain situations, but not in some others.

**Reliability** means dependability or
trustworthiness. It is the degree to
which a test consistently measures whatever it measures.

Reliability is expressed numerically, usually as a coefficient; a high coefficient indicates high reliability. High reliability indicates minimum error variance; if a test has high reliability, then the effect of errors of measurement has been reduced.

A valid test is always reliable but a reliable test is not necessarily valid. In other word, if a test is measuring what it is supposed to be measuring, it will be reliable and do so every time, but a reliable test can consistently measure the wrong thing and be invalid.

The reliability is easier than validity to be assessed. Test-retest, equivalent-forms, and split-half reliability are all determined through correlation.

Test-retest reliability is the degree to which scores are consistent over time.

Reliability can also be expressed in terms of the standard error of measurement. Standard error of measurement (degree of variance) is an estimate of how often you can expect errors of a given size. A small standard error of measurement indicates high reliability and a large standard error of measurement indicates low reliability.

**Design and Procedure**

*Basic causal-comparative research design*

The basic causal-comparative design involves selecting two groups differing on some independent variable and comparing them on some dependent variable.

Case A: (E) – (X) – (O)

(C) -- (O) One group is with different experiences, the other is not

Case B: (E) – (X_{1})--(O)

(C) – (X_{2}) –(O) Two groups with two different
experiences

***Symbols: (E) – Experimental group, (X) – Independent variable

(C) – Control group, (O) – Dependent variable

Three types of control procedures that can be used in a causal-comparative study

Since the randomization is not possible in causal-comparative studies (the groups already exist), three types of control procedures usually adopted in the study to insure equality of groups.

1) ** Matching**: If a
researcher match on IQ, then a subject in one
group with an IQ at or near 140 would have a match in the other group.

2) ** Comparing homogeneous groups or subgroups**: If IQ were an
identified extraneous variable, the researcher might limit groups to contain
only subjects with IQ between 85 and 115 (average IQ). Or, each group might be divided into high
(116 and above), average (85-115) and low (84 and below) IQ subgroups.

3) ** A statistic method, Analysis of covariance (ANCOVA)**: A statistic method to be used to equate
groups on one or more variables.

** **

*Common experimental research design*

1) Pre-experimental Designs

X -- O One-short case study

O-- X -- O One group pretest-post test design

X_{1} -- O / X_{2} -- O
Static group comparison

2) True experimental designs

R -- O -- X_{1} -- O

R -- O -- X_{2} -- O Pretest-posttest control group design

R -- X1 -- O

R -- X2 -- O Posttest only control group design

3) Quasi-experimental group designs

O -- X1 -- O

O -- X2 -- O Nonequivalent control group design

O -- O -- O -- O -- X -- O -- O -- O -- O Time series design

X1 -- O -- X2 -- O -- X3 -- O

X3 -- O -- X1 -- O -- X2 -- O

X2 -- O -- X3 -- O -- X1 -- O Counterbalanced design

***Symbols: (R) – Randon assignment of subjects to group

(X) – Independent variable (O) – Dependent variable

Five ways to control extraneous variables

1) ** Randomization** is effective in creating equivalent and
representative groups that are essentially the same on all relevant variables
thought of by the researcher.

2) ** Matching** is a techniques for equating groups on one or
more variables the researcher has identified as being highly related to
performance on the dependent variable.

3) ** Comparing homogeneous groups or subgroups** to control the
extraneous variables.

4) ** Using subjects as their own controls** involves exposing
the same group to the different treatments, one treatment at a time.

5) ** Analysis of covariance** is a statistical method for
equating randomly formed groups on one or more variables.

Three types of replication involved in single-subject research

1) ** Direct replication** refers to replication by the same
investigator, with the same subject or with different subjects, in a specific
setting.

2) ** Systematic replication** refers to replication which
follows direct replication, and which involves different investigators,
behaviors, and settings.

3) ** Clinical replication** involves the development of a
treatment package, composed of two or more interventions that have been found
to be effective individually, designed for persons with complex behavior
disorders.

Reference:

Gay, L. R. (1996). Educational research: Competencies for analysis and application. Upper Saddle River, NJ: Merrill.