Abstract

Improving the quality of Education leads to the improvement of the social and economic development rates of individuals and countries. Furthermore, few studies seek to identify effective practices in the pedagogical and management areas that can direct educational actors in their actions. Thus, this study aims to identify indicators and goals for increasing school performance in the subject of Portuguese in municipal lower secondary Education. These practices were identified based on the analysis of Brazilian municipal Schools of High Performance. Municipal schools were selected and grouped by socioeconomic level to measure efficiency through the two-stage DEA tool. The results show that the increase in current expenditure is not always accompanied by the improvement in the results. In addition, 45 indicators and goals that can increase the performance of schools in reading and writing teaching were found. They are related to the various actors of the educational process, namely: students, teachers and principals. Of these actors, some were related to the Principal’s management skills, reinforcing his/her role in promoting the quality of teaching.

Indicators and Goals; School Management; Reading Performance; Data Envelopment Analysis; Ideb

Resumo

Melhorar a qualidade da Educação leva à melhoria das taxas de desenvolvimento social e econômico de indivíduos e países. Poucos estudos buscam identificar práticas efetivas nas áreas pedagógica e gerencial que possam direcionar os atores educacionais em suas ações. Assim, esse estudo tem como objetivo identificar indicadores e metas para o aumento do desempenho escolar na disciplina de português no Ensino Fundamental municipal - nível II. Essas práticas foram identificadas com base na análise das escolas municipais brasileiras de alto desempenho. As escolas foram selecionadas e agrupadas por nível socioeconômico para mensurar a eficiência, por meio da ferramenta DEA de dois estágios. Além disso, foram encontrados 45 indicadores e seus objetivos, que podem aumentar o desempenho das escolas no ensino de leitura de escrita. Eles estão relacionados aos vários atores do processo educacional, como: estudantes, professores e diretores. Alguns indicadores são relacionados às habilidades de gerenciamento do Diretor, reforçando ainda mais seu papel na promoção da qualidade do Ensino.

Indicadores e Metas; Gestão Escolar; Desempenho de Leitura; Análise de Envelope de Dados; Ideb

Resumen

Mejorar la calidad de la Educación conduce a la mejora de las tasas de desarrollo social y económico de individuos y países. En otro ejemplo, pocos estudios buscan identificar prácticas efectivas en las áreas pedagógicas y de gestión que puedan dirigir a los actores educativos en sus acciones. Por lo tanto, este estudio tiene como objetivo identificar indicadores y objetivos para aumentar el rendimiento escolar en la disciplina de portugués en la Escuela Primaria municipal - nivel II. Estas prácticas se identificaron embasadas en el análisis de las escuelas primarias municipales brasileñas de alto rendimiento. La escuela primaria municipal - nivel II fue seleccionada y agrupada por nivel socioeconómico para medir la eficiencia a través de la herramienta DEA de dos etapas. Los resultados muestran que el aumento de las inversiones no siempre va acompañado de una mejora en los resultados. Además, se encontraron 45 indicadores y sus objetivos, que pueden aumentar el rendimiento de las escuelas en la enseñanza de lectura y escritura. Se relacionan con los distintos actores del proceso educativo, a saber: estudiantes, docentes y directores. De estos actores, algunos se relacionaron con las habilidades de gestión del director, reforzando su papel en la promoción de la calidad de la Enseñanza.

Indicadores y Metas; Gestión Escolar; Rendimiento de Lectura; Análisis Envolvente de Datos; Ideb

1 Introduction

Education has a transforming role in the development of citizens due to a series of factors, such as: the economic development of nations ( CARVALHO; SOUSA, 2014CARVALHO, L. D. B.; SOUSA, M. C. S. Eficiência das escolas públicas urbanas das regiões nordeste e sudeste do Brasil: uma abordagem em três estágios. Estudos Econômicos , São Paulo, v. 44, n. 4, p. 649-684, Oct./Dec. 2014. https://doi.org/10.1590/S0101-41612014000400001
https://doi.org/10.1590/S0101-4161201400... ), determination and distribution of income ( NEAMTU, 2015NEAMTU, D. M. Education, the economic development pillar. Procedia: Social and Behavioral Sciences , Amsterdam, v. 180, p. 413-420, May 2015. https://doi.org/10.1016/j.sbspro.2015.02.138
https://doi.org/10.1016/j.sbspro.2015.02... ) and the improvement of social markers ranging from, among others, crime rates, population health conditions, social cohesion, life expectancy and infant mortality rates ( CARVALHO; SOUSA, 2014CARVALHO, L. D. B.; SOUSA, M. C. S. Eficiência das escolas públicas urbanas das regiões nordeste e sudeste do Brasil: uma abordagem em três estágios. Estudos Econômicos , São Paulo, v. 44, n. 4, p. 649-684, Oct./Dec. 2014. https://doi.org/10.1590/S0101-41612014000400001
https://doi.org/10.1590/S0101-4161201400... ).

Such aforementioned factors, hand in hand with high values of current expenditure by governments and families, have raised the interest of economic and administration science in the Education area, generating new studies (GOMES et al ., 2016; DUMCIUVIENE, 2015DUMCIUVIENE, D. The impact of education policy to country economic development. Procedia: Social and Behavioral Sciences , Amsterdam, v. 191, p. 2427-2436, June 2015. https://doi.org/10.1016/j.sbspro.2015.04.302
https://doi.org/10.1016/j.sbspro.2015.04... ; MELHUISH, 2013MELHUISH, E. Efeitos de longo prazo da educação infantil: evidências e política. Cadernos de Pesquisa , São Paulo, v. 43, n. 148, p. 124-149, Jan./Apr. 2013. https://doi.org/10.1590/S0100-15742013000100007
https://doi.org/10.1590/S0100-1574201300... ; NEAMTU, 2015NEAMTU, D. M. Education, the economic development pillar. Procedia: Social and Behavioral Sciences , Amsterdam, v. 180, p. 413-420, May 2015. https://doi.org/10.1016/j.sbspro.2015.02.138
https://doi.org/10.1016/j.sbspro.2015.02... ; SALGADO JUNIOR; NOVI, 2015; SALGADO JUNIOR et al ., 2016), which have focused on the debate related to “efficiency in Education” ( WITTE; LÓPEZ-TORRES, 2015WITTE, K.; LÓPEZ-TORRES, L. Efficiency in education: a review of literature and a way forward. Journal of the Operational Research Society , Birmingham, v. 68, n. 4, p. 339-363, Mar. 2015. ). The demand for such studies emerges from the necessity of governments to balance their expenditure in the area with other essential public services, such as health and transportation (JOHNES et al ., 2017).

When evaluating Education, it is necessary to understand that quality is the most important variable to explain economic development, outweighing quantity (SALGADO JUNIOR; NOVI, 2014). The difference in economic performance among countries is related to differences in their Education levels ( MERCAN; SEZER, 2014MERCAN, M.; SEZER, S. The effect of education expenditure on economic growth: the case of turkey. Procedia: Social and Behavioral Sciences , Amsterdam, v. 109, p. 925-930, Jan. 2014. https://doi.org/10.1016/j.sbspro.2013.12.565
https://doi.org/10.1016/j.sbspro.2013.12... ). Similarly, due to internal quality differences, this phenomenon may be observed within the same country or Educational system (WU et al ., 2017).

The concept of quality in the educational area addresses structures, processes and educational outcomes ( DAVOK, 2007DAVOK, D. F. Qualidade em educação. Avaliação: Revista da Avaliação da Educação Superior (Campinas) , Sorocaba, v. 12, n. 3, p. 505-513, Sept. 2007. https://doi.org/10.1590/S1414-40772007000300007
https://doi.org/10.1590/S1414-4077200700... ). In this sense, two distinct dimensions are approached: (i) from the institutional point of view, referred to educational quality; (ii) and from the point of view on generated results at the central main agent, the student. In the first sense, efficiency, efficacy, effectiveness and relevance of educational systems and institutions are contemplated; and in the second sense, the development of learning skills on predicted contents, knowledge of scientific-literary culture, and technical knowledge for the labor market and social transformation are considered.

Dourado and Oliveira (2009)DOURADO, J. F.; OLIVEIRA, J. F. A qualidade da educação: perspectivas e desafios. Cadernos Cedes , Campinas, v. 29, n.78, p. 201-215, May/Aug. 2009. https://doi.org/10.1590/S0101-32622009000200004
https://doi.org/10.1590/S0101-3262200900... acknowledge the recent advance in Brazil to access and coverage, especially in elementary school; however, it is necessary to improve learning process in an efficient way that improves the quality of teaching. Education is transversal to pedagogical, economic, social, cultural and political dynamics of a given society, according to Darling-Hammond and Ascher (1991)DARLING-HAMMOND, L.; ASCHER, C. Creating accountability in big city schools . New York: National Center for Restructuring Education, Schools and Teaching, 1991. (Urban Diversity Series 102). and Dourado et al. (2007). These authors claim that influencing factors are: from the social point of view, income concentration, social inequality, Education as a right, and institutional issues such as school organization and management that interfere with working conditions, school management processes, curricular dynamics, teacher training and teacher professionalization.

“There is a certain consensus that the school quality assessment theme needs to be addressed and that quality measurement must necessarily involve the use of different tools”¹ 1 Translated by authors. ( GOUVEIA; SOUZA, 2013GOUVEIA, A. B.; SOUZA, A. R. Efetividade da política para o ensino fundamental em municípios brasileiros. Cadernos de Pesquisa , São Paulo, v. 43, n. 150, p. 836-855, Sept./Dec. 2013. https://doi.org/10.1590/S0100-15742013000300006
https://doi.org/10.1590/S0100-1574201300... , p. 841). The evaluation of the quality of Education remains a controversial issue in the literature and has been carried out through large-scale evaluations, which generally evaluate the performance of students in specific subjects, such as mathematics and reading ( FERNANDES; GREMAUD, 2009FERNANDES, R.; GREMAUD, A. P. Qualidade da educação: avaliação, indicadores e metas. In: VELOSO, F. et al. (orgs.). Educação b ásica no Brasil: construindo o país do futuro. Rio de Janeiro: Elsevier, 2009. p. 213-238. ).

Among the criticisms of current large-scale evaluation models, it is possible to mention the limited focus, which restricts the evaluation of a multi-context and facets system to some subjects to the detriment of other expected school objectives, such as ethics and the development of some non-cognitive skills as well as the non-identification of other social development factors related to Education ( FERNANDES; GREMAUD, 2009FERNANDES, R.; GREMAUD, A. P. Qualidade da educação: avaliação, indicadores e metas. In: VELOSO, F. et al. (orgs.). Educação b ásica no Brasil: construindo o país do futuro. Rio de Janeiro: Elsevier, 2009. p. 213-238. ).

Brazil has not presented satisfactory results in those evaluations. In the Program for International Students Assessment (Pisa), Brazil presented, in the 2015 edition, an average of 407 points in reading in comparison with the average of 493 points from other evaluated countries (OECD, 2015).

Inside national scope, the Development of Basic Education Index (Ideb) is the main quality indicator available. It is composed by two components: approval index and performance on the Prova Brasil (Brazil Exam). The introduction of Ideb by the federal government represented a significant change in the way Education has been monitored in Brazil. Passing through Ideb, the evolution of educational services offered inside the country raised, this being possible thanks to tracking the results by state, municipality and school, providing a comparative value in relation to goals, and also comparing to other schools and municipalities ( ALVES; SOARES, 2013ALVES, M. T. G.; SOARES, J. F. Contexto escolar e indicadores educacionais: condições desiguais para a efetivação de uma política de avaliação educacional. Educação e Pesquisa , São Paulo, v. 39, n. 1, p. 177-194, Jan./Mar. 2013. https://doi.org/10.1590/S1517-97022013000100012
https://doi.org/10.1590/S1517-9702201300... ).

In the Brazil Exam, the largest instrument for evaluating Brazilian basic Education, the average score in the subject of Portuguese language, in the 2015 edition, was 252 points, which was below the score obtained by students in 1995, when 256 average points were reached in the final years of elementary school ( INEP, 2016INSTITUTO NACIONAL DE ESTUDOS E PESQUISAS EDUCACIONAIS ANÍSIO TEIXEIRA – INEP. Inep apresenta resultados do Saeb/prova Brasil 2015. Notícias Saeb , Sept. 8, 2016. Available from: http://portal.inep.gov.br/artigo/-/asset_publisher/B4AQV9zFY7Bv/content/inep-apresenta-resultados-do-saeb-prova-brasil-2015/21206. Access in: 2018 Feb. 5.
http://portal.inep.gov.br/artigo/-/asset... ). In spite of the growing current expenditure in recent years, this diagnosis demonstrates the need for improvements in the country’s educational process.

Despite the strong connection identified among extra-school factors, such as the socioeconomic level of the student and family, and their educational results (ALVES et al., 2013; MATOS et al., 2017), several studies pinpoint the importance of the “school effect”, which is a part of the result of students that can be attributed to the pedagogical and management practices adopted in schools ( ALVES; SOARES, 2007ALVES, M. T. G.; SOARES, J. F. Efeito-escola e estratificação escolar: o impacto da composição de turmas por nível de habilidade dos alunos. Educação em Revista , Belo Horizonte, n. 45, p. 25-59, June 2007. https://doi.org/10.1590/S0102-46982007000100003
https://doi.org/10.1590/S0102-4698200700... ; BLOOM et al., 2015; GONÇALVES; FRANCE, 2013; GOUVEIA; SOUZA, 2013GOUVEIA, A. B.; SOUZA, A. R. Efetividade da política para o ensino fundamental em municípios brasileiros. Cadernos de Pesquisa , São Paulo, v. 43, n. 150, p. 836-855, Sept./Dec. 2013. https://doi.org/10.1590/S0100-15742013000300006
https://doi.org/10.1590/S0100-1574201300... ; SALGADO JUNIOR; NOVI, 2015; SALGADO JUNIOR et al., 2016).

Several studies have indicated the relevance of intra-school factors in student performance, such as administrative management (BLOOM et al., 2015), low remuneration of professionals, lack of training and inadequate work environment (AMÂNCIO-VIEIRA et al., 2015), principal profile in pedagogical results ( DHUEY; SMITH, 2014DHUEY, E.; SMITH, J. How important are school principals in the production of student achievement? Canadian Journal of Economics , Vancouver, v. 47, n. 2, p. 634-663, Jan. 2014. https://doi.org/10.1111/caje.12086
https://doi.org/10.1111/caje.12086... ), innovative didactic project ( DOLZ, 2016DOLZ, J. As atividades e os exercícios de língua: uma reflexão sobre a engenharia didática. Delta: Documentação de Estudos em Lingüística Teórica e Aplicada , São Paulo, v. 32, n. 1, p. 237-260, Jan./Apr. 2016. https://doi.org/10.1590/0102-4450321726287520541
https://doi.org/10.1590/0102-44503217262... ), offer of classes during school holidays to students at disproof risk (VANDECANDELAERE et al., 2016) and emotional factors ( PARISOTTO; RINALDI, 2016PARISOTTO, A. L. V.; RINALDI, R. P. Ensino de língua materna : dificuldades e necessidades formativas apontadas por professores na educação fundamental. Educar em Revista , Curitiba, n. 60, p. 261-276, Apr./June 2016. https://doi.org/10.1590/0104-4060.45125
https://doi.org/10.1590/0104-4060.45125... ).

The identification of these practices is fundamental, as they can contribute to the improvement of the performance of the educational service and the results of students, allowing a systemic view of Education that encompasses pedagogical aspects as well as those related to management activities (SALGADO JUNIOR; NOVI, 2015).

In addition, since Education is a process of aggregation of knowledge, the studies that focus on improving elementary school, which is one of the most important level of Education, are essential as the experience of a child at this stage brings relevant consequences to the rest of his/her academic life ( MELHUISH, 2013MELHUISH, E. Efeitos de longo prazo da educação infantil: evidências e política. Cadernos de Pesquisa , São Paulo, v. 43, n. 148, p. 124-149, Jan./Apr. 2013. https://doi.org/10.1590/S0100-15742013000100007
https://doi.org/10.1590/S0100-1574201300... ). This Education level is also relevant due to its representativeness in the Brazilian educational system: data from the 2013 School Census indicated that 29.7 million out of the 50.5 million students enrolled in basic Education in 2012, 68.2%, studied in the municipal educational systems (REGALO et al., 2016). Among the competencies acquired in the school, reading and writing skills are of paramount importance, as students who have better reading skills, perform better in other disciplines (OLIVEIRA et al ., 2008). Apart from the benefits in the educational field, proper literacy provides the individual with economic gains, as people with higher literacy earn more than those with lower literacy levels (BRAY et al., 2014).

In light of the above, the present work aims to identify the indicators and goals for the increase in school performance in the subject of Portuguese language in municipal lower secondary Education. By using such parameters, some evidences show the improvement of students’ performance in reading and writing in the final years of elementary school.

2 Methodological aspects

In order to improve school performance and identify its indicators and goals, the methodology used was divided into two parts. The first consisted of the separation of schools that were studied according to the level of efficiency in relation to knowledge aggregation as well as the knowledge of the students of lower secondary Education, which is also called school effect. Once the level of efficiency was defined, the next move dealt with identifying the factors related to school performance and its goals.

Barra and Zotti (2016)BARRA, C.; ZOTTI, R. Measuring efficiency in higher education: an empirical study using a bootstrapped data envelopment analysis. International Advances in Economic Research , Atlanta, v. 22, n. 1, p. 11-33, Feb. 2016. https://doi.org/10.1007/s11294-015-9558-4
https://doi.org/10.1007/s11294-015-9558-... indicate the use of non-parametric methods for the educational sector as more appropriate because, different from the parametric techniques, they do not require any knowledge of the theoretical boundary of production. Another advantage is that “non-parametric methods can handle multiple inputs and outputs in a simple manner, while most stochastic approaches require the choice of a single explanatory variable” ( Figure 1 ) ( WITTE; LÓPEZ-TORRES, 2015WITTE, K.; LÓPEZ-TORRES, L. Efficiency in education: a review of literature and a way forward. Journal of the Operational Research Society , Birmingham, v. 68, n. 4, p. 339-363, Mar. 2015. , p.15).

Thus, the authors chose to use the Data Envelopment Analysis to measure efficiency. The technique allows a relatively efficient comparison between the groups of schools that were studied, which proved to be a necessary procedure for the development of this research (SALGADO JUNIOR; NOVI, 2015).

The aim of the DEA is to allow the comparison of Decision Making Units (DMU), which perform similar tasks obtaining different results in the relationship between the number of input used and the amount of output produced (MEZA et al., 2005). The efficiency frontier resulting from the application of the DEA technique ( Figure 2 ) indicates the efficient DMU that can be studied and compared to find the best practices. In addition, the technique also shows how distant other DMU are from those so-called efficient (FLACH et al ., 2017).

In light of this, it is essential to define the efficiency model in order to find the correct relationship with their respective explanatory factors. The proposed efficiency model aims to identify schools that have managed to aggregate more performance in the results of the subject of Portuguese-language in the Prova Brasil in relation to the same group of students.

For this, this research takes as input the average per school of the performance of students in Portuguese language in the Prova Brasil , taken at the end of the fifth year. The use of this input corrects one of the main problems of research on the subject, since most studies ignore the previous results obtained by students, resulting in the impossibility of estimating efficiency impartially ( WITTE; LÓPEZ-TORRES, 2015WITTE, K.; LÓPEZ-TORRES, L. Efficiency in education: a review of literature and a way forward. Journal of the Operational Research Society , Birmingham, v. 68, n. 4, p. 339-363, Mar. 2015. ). As output, the average performance was used in this evaluation at the end of the ninth year as well as the performance indicator of each school in this period, which is an index that presents the approval rate of students among the evaluations as shown in Chart 1 and Figure 3 .

Thus, the proposed model seeks to evaluate the same cycle of students of each school, according to Figure 3 , to verify those who aggregated the highest performance, considering that, for the most part, students remained in the same institution throughout all the years of elementary school.

Two models considered classic are CCR and BCC. The CCR model (CHARNES et al ., 1978), “considers constant returns to scale, that is, any variation in input produces proportional variation in output” (MEZA et al., 2005, p. 2525). The BCC model considers variable returns to scale, being more appropriate in relation to CCR to produce more efficient DMU (BANKER et al. , 1984).

According to Flach et al. (2017), output guidance is the most appropriate because the objective of the public service is not to reduce input, but rather to have the best application to obtain the best results to the society. In this study, since the goal is to improve the student’s performance in the output test of the model ( Prova Brasil of the 8th grade), and not reduce his/her entry grade ( Prova Brasil of the 4th grade), the orientation to output will be used. Furthermore, in the hypotheses in which the establishment of proportionality between input and output is not possible, as in this study, it is recommended the adoption of the BCC model (SALGADO JUNIOR; NOVI, 2015). The model formulation can be represented as follows:

Subject to restrictions:

ur, vi ≥ 0

j = 1, ... , s

r = 1, ... , m

i = 1, ... , n

Where Y _ij and X _ij are the products and input of j-th DMU, u _r and v _i represent the weights (weighting coefficients or relative importance of each variable) to be determined by the solution of the problem. The u _r and v _i weights obtained are corresponding to the current DMU. This process is repeated for each of the DMU, obtaining different values for u _r and v _i .

However, since the DEA is a non-parametric technique, it is not susceptible to statistical inference, presenting great dependence on the quality of the sample and the specification of the efficiency model adopted ( BARRA; ZOTTI, 2016BARRA, C.; ZOTTI, R. Measuring efficiency in higher education: an empirical study using a bootstrapped data envelopment analysis. International Advances in Economic Research , Atlanta, v. 22, n. 1, p. 11-33, Feb. 2016. https://doi.org/10.1007/s11294-015-9558-4
https://doi.org/10.1007/s11294-015-9558-... ; MARQUES; SILVA, 2006MARQUES, R. C.; SILVA, D. Inferência estatística dos estimadores de eficiência obtidos com a técnica fronteira não paramétrica de DEA: uma metodologia de bootstrap. Investigação Operacional , Lisboa, v. 26, p. 89-110, 2006. ). An alternative to this scenario is the implementation of the bootstrap methodology ( EFRON, 1979EFRON, B. Bootstrap methods: another look at the Jackknife. Annals of Statistics , Beachwood, v. 7, n. 1, p. 1–26, Jan. 1979. https://doi.org/10.1214/aos/1176344552
https://doi.org/10.1214/aos/1176344552... ) to DEA estimators. It allows statistical inference ( MARQUES; SILVA, 2006MARQUES, R. C.; SILVA, D. Inferência estatística dos estimadores de eficiência obtidos com a técnica fronteira não paramétrica de DEA: uma metodologia de bootstrap. Investigação Operacional , Lisboa, v. 26, p. 89-110, 2006. ) and the achievement of more reliable estimates of efficiency indexes and confidence intervals ( DELGADO; MACHADO, 2007DELGADO, V. M. S.; MACHADO, A. F. Eficiência das escolas públicas estaduais de Minas Gerais. Pesquisa e Planejamento Econômico , Rio de Janeiro, v. 37, n. 3, p. 427-464, Dec. 2007. , p. 435).

Because of this, for this study, we opted for the application of the BCC DEA with bootstrap, in which “estimates of corrected frontier by adjusted polarization are obtained, indicating the sensitivity of efficiency scores in relation to the sampling variations of the estimated frontier” ( BARRA; ZOTTI, 2016BARRA, C.; ZOTTI, R. Measuring efficiency in higher education: an empirical study using a bootstrapped data envelopment analysis. International Advances in Economic Research , Atlanta, v. 22, n. 1, p. 11-33, Feb. 2016. https://doi.org/10.1007/s11294-015-9558-4
https://doi.org/10.1007/s11294-015-9558-... , p. 13). Finally, the PIM DEA software, version 3.2, was used to calculate the efficiency scores.

For the application of the DEA technique in the proposed model, data from the municipal elementary schools of Prova Brasil were extracted from the Inep website; Inep’s test is applied every 2 years. The results are only available for consulting at school level; therefore, the data of the average grades in the subject of Portuguese language of the students by school are used. From the 2011 base of Prova Brasil , the 5th year and 2015 base grades were used as well as the Performance Indicator of the Ideb of 2015. The use of data from the base years 2011 and 2015 was adopted to measure the evolution of the performance of the same cycle of students, since there is a high probability that those who took the test in the 9th year in 2015 are the same who performed it in 2011, during the 5th year. Data extraction was performed by developing a SQL program using Postgre SQL software.

The initial database had the performance data of 40,957 municipal elementary schools from all over the country. Initially a filter was performed, remaining only those schools that offered both the early years and the final years of elementary school, admitting that, in general, students tend to continue their studies at the same institution. Following the procedures, there was a new restriction in order to maintain only schools that had all the necessary data for the execution of the DEA. Then, schools that had obtained a performance indicator of 2015 lower than 0.50 were eliminated from the base, keeping 9,120 schools.

The DEA technique presents as a restriction the inability to distinguish the inefficiency attributed to the poor management practices from those resulting from socioeconomic differences or other factors that are not under the managers control ( CARVALHO; SOUSA, 2014CARVALHO, L. D. B.; SOUSA, M. C. S. Eficiência das escolas públicas urbanas das regiões nordeste e sudeste do Brasil: uma abordagem em três estágios. Estudos Econômicos , São Paulo, v. 44, n. 4, p. 649-684, Oct./Dec. 2014. https://doi.org/10.1590/S0101-41612014000400001
https://doi.org/10.1590/S0101-4161201400... ). Thus, the homogeneity of DMU is presented as pre-supposed for its use, that is, the units must perform equal tasks, with similar objectives, acting under the same market conditions and having autonomy for decision-making (MEZA et al., 2005). Ensuring their homogeneity is an essential item for the efficiency scale obtained to reflect only the differences between the practices adopted by schools and not the interference of external factors.

Thus, initially schools need to be separated according to socioeconomic level - SEL, to allow a fairer comparison among them ( ANDRADE; SOARES, 2008ANDRADE, R. J.; SOARES, J. F. O efeito da escola básica brasileira. Estudos em Avaliação Educacional , São Paulo, v. 19, n. 41, p. 378-406, 2008. https://doi.org/10.18222/eae194120082067
https://doi.org/10.18222/eae194120082067... ), since this factor has been identified in educational research as one of the main influencers of student performance ( ALVES; SOARES, 2007ALVES, M. T. G.; SOARES, J. F. Efeito-escola e estratificação escolar: o impacto da composição de turmas por nível de habilidade dos alunos. Educação em Revista , Belo Horizonte, n. 45, p. 25-59, June 2007. https://doi.org/10.1590/S0102-46982007000100003
https://doi.org/10.1590/S0102-4698200700... ; HAELERMANS; RUGGIERO, 2013HAELERMANS, C.; RUGGIERO, J. Estimating technical and allocative efficiency in the public sector: A nonparametric analysis of dutch schools. European Journal of Operational Research , Amsterdam, v. 227, n. 1, p. 174-181, May 2013. https://doi.org/10.1016/j.ejor.2012.12.004
https://doi.org/10.1016/j.ejor.2012.12.0... ).

To ensure homogeneity among DMU, schools were classified into seven bands, according to the SEL proposed by Alves, Soares and Xavier (2014), with the sample having 3,330 schools of SEL 5 and 6, which were used to generate the efficiency ranking using the DEA technique.

The following were the data of the contextual questionnaires of Prova Brasil (four context questionnaires: about the school, the teacher, the principal and the students) and the data of the School Census for each school. Data on the school and the principal were extracted from the bases of the School Census and the Prova Brasil of 2013, to identify the structure available during the period in which the student was studying as well as the characteristics of the principal and the management actions implemented by him/her, whose effect can be observed over a longer period. For students and teachers, we opted for the use of the 2015 database, since such data more accurately portray the characteristics of these actors at the time of the last edition of Prova Brasil , understanding that these characteristics have a direct influence on the result of the evaluation; 1,392 variables were obtained in these databases.

Due to the large volume of variables, a two-step analysis was performed: initially a quintile analysis was performed to identify those with significant differences between the groups and then a second stage of DEA, which is the Multiple Linear Regression, was used.

For each of the variables, normality test was performed to guide the next stages of the analysis. Then, the averages and/or medians of these variables were obtained, and, through the Mann-Whitney test, those with significant differences between the lower and upper quintiles were identified.

Subsequently, those that appeared in duplicity between the bases and those inversely significant dichotomous were removed since they offered the same response, opting for the maintenance of only one of the variables. Some variables were also maintained, which, despite not having a difference between high and low-performance schools, were understood as relevant and were also the object of other studies identified during the survey of the theoretical framework. These were classified as “neutral” and are the ones whose average or median showed no statistically significant difference between the upper and lower quintiles. After this last analysis, 162 variables remained.

From these variables, the second stage of DEA was performed, through Multiple Linear Regression. This technique has been widely applied to research in Education ( ALVES; SOARES, 2013ALVES, M. T. G.; SOARES, J. F. Contexto escolar e indicadores educacionais: condições desiguais para a efetivação de uma política de avaliação educacional. Educação e Pesquisa , São Paulo, v. 39, n. 1, p. 177-194, Jan./Mar. 2013. https://doi.org/10.1590/S1517-97022013000100012
https://doi.org/10.1590/S1517-9702201300... ; AMÂNCIO-VIEIRA et al., 2015; ANDRADE; SOARES, 2008ANDRADE, R. J.; SOARES, J. F. O efeito da escola básica brasileira. Estudos em Avaliação Educacional , São Paulo, v. 19, n. 41, p. 378-406, 2008. https://doi.org/10.18222/eae194120082067
https://doi.org/10.18222/eae194120082067... ), and allows to identify the existence of a dependent relationship between the dependent variable, that is, the one that one wishes to explain – in the case of this study, the efficiency score of each school generated by the DEA technique – and the independent ones, that is, those that may be related to this performance; in this work, independent ones are related to those available in the bases of the School Census and the Prova Brasil (CORRAR et al., 2009, p. 132).

Multiple Linear Regression Equation:

Where:

Y = Dependent Variable (DEA BCC score);

β₀ = Intercept (Value of Y when X = 0);

X₁, X₂, ... X_n = independent variables (variables identified in the quintile analysis);

β₁, β₂, ... β_n = regression parameters;

ε = residual or regression error.

The method used was sequential search, with stepwise estimation. According to Corrar et al. (2009, p. 160), “this method performs an initial estimate with a set of independent variables and adds or eliminates variables until they achieve the best measure within the criterion used”. For this, the IBM SPSS Statistics version 22 software was used. The Figure 4 summarizes the steps of the method.

Based on this two-stage DEA analysis, it was possible to both measure the level of efficiency of schools (first step) and identify factors related to their performance. From the exploration of the method, it was possible to determine the goals of each identified indicator. The constant value of the upper quintile and its respective trend was set as a goal. Thus, it is possible for each school to evaluate its own individual performance in relation to each of the indicators related to reading and writing. The next section details the search results.

3 Results

The selection of the statistical techniques is linked to the type of distribution of the data. Initially we present the analysis of the BCC DEA scores to verify whether they presented normal distribution or not.

The visual analysis of Graph 1 seems to indicate that the DEA BCC score does not present normal distribution, which can be confirmed by the Kolmogorov-Smirnov statistical test, according to Table 1 , since its Sig was 0.000 (Sig < 0.05).

The following is the descriptive statistics for the quintiles with DEA score, performance in Portuguese language in Prova Brasil , in their editions of 2011 and 2015 and Performance Indicator of 2015 ( Table 2 ).

A quintile analysis was performed to identify those ones that presented a statistically significant average or median difference between the upper quintile (schools considered of high performance in the DEA ranking) and the lower quintile (schools considered of low performance in the DEA ranking).

From the 161 variables identified in the quintile analysis, the analysis was performed using Multiple Linear Regression. The stepwise model pointed to a result composed of 39 selected independent variables. This model, due to the complexity involved in the educational service and the influence of numerous external factors on the students’ results, obtained a R² of 0.248, presenting limited explanatory power, according to Table 3 . However, if the technical assumptions are met, we can use its results to reinforce the analysis on the items highlighted by it.

According to Corrar et al. (2009), the main assumptions for regression analysis are normality and homocedasticity of the residuals. The analysis of the histogram of the residuals ( Graph 2 ) seems to indicate normality, which is confirmed by the Kolmogorov-Smirnov test ( Table 4 ).

The verification of linear relationships between the variables can be performed through statistical tests such as the Variance Inflation Factor - VIF and Tolerance (CORRAR et al., 2009). For these indexes, it can be observed in Table 5 that there is no multicollinearity among the independent variables that remained in the model, since both, VIF and tolerance, are within the acceptable ranges.

Thus, the model complies with the statistical assumptions of the analysis, assisting in the identification of variables with greater explanatory potential among those initially selected, which are identified in Table 5 .

Once regression was performed, it was possible to identify, at a level of 5% confidence, which variables were significant to explain the students’ school performance in the Ideb evaluations. The variables were classified according to their effect in schools – negative, positive or neutral. Variables with negative effect were considered those that, presenting statistically significant difference, were more present in low-performance schools; and positive, those who, also presenting statistically significant difference, had higher frequency in schools considered efficient by the model. For each variable, the constant value of the upper quintile and its respective trend was set as a goal.

In addition to the identification of indicators, it is also essential to define the goals for each of these, allowing each school to evaluate its individual performance compared to the others in each aspect addressed in this study.

School A is located in a municipality with approximately 15,000 inhabitants with GDP per capita of R$ 16,058.64 ( IBGE, 2016INSTITUTO BRASILEIRO DE GEOGRAFIA E ESTATÍSTICA – IBGE. Censo demográfico 2015 . Rio de Janeiro, 2016. Available from: https://cidades.ibge.gov.br. Access in: Oct. 20, 2018.
https://cidades.ibge.gov.br... ). It is located in a country town of São Paulo state, close to important centers such as Piracicaba and Limeira cities. The municipality also does not have private elementary schools.

The municipality referring to school B has approximately 137,000 inhabitants with GDP per capita of R$ 35,223.10 ( IBGE, 2016INSTITUTO BRASILEIRO DE GEOGRAFIA E ESTATÍSTICA – IBGE. Censo demográfico 2015 . Rio de Janeiro, 2016. Available from: https://cidades.ibge.gov.br. Access in: Oct. 20, 2018.
https://cidades.ibge.gov.br... ), and is located in a country town of São Paulo state, exactly 60 kilometers from Campinas and Limeira cities. The municipality has a strong performance in citrus, besides housing large industrial enterprises in the metal mechanics sector and the paper industry. The municipality has a large network of private primary schools.

The school C is located in a municipality with approximately 29,000 inhabitants with GDP per capita of R$ 31,168.75 ( IBGE, 2016INSTITUTO BRASILEIRO DE GEOGRAFIA E ESTATÍSTICA – IBGE. Censo demográfico 2015 . Rio de Janeiro, 2016. Available from: https://cidades.ibge.gov.br. Access in: Oct. 20, 2018.
https://cidades.ibge.gov.br... ), and is located in a country town of São Paulo state, with 16 kilometers from the city of Piracicaba. The city has municipal, state and private schools to offer basic education.

Finally, the municipality of school D has about 51,000 inhabitants with GDP per capita of R$ 30,453.74 ( IBGE, 2016INSTITUTO BRASILEIRO DE GEOGRAFIA E ESTATÍSTICA – IBGE. Censo demográfico 2015 . Rio de Janeiro, 2016. Available from: https://cidades.ibge.gov.br. Access in: Oct. 20, 2018.
https://cidades.ibge.gov.br... ), and is located in a country town of São Paulo state, 130 km distant from Ribeirão Preto city and 160 km from Campinas city. The government is the largest employer of the municipality, being also responsible for most of the local economy, followed by industry and agriculture, respectively. The municipality has a network of private primary schools.

Thus, from the results found in linear regression, the results of high-performance schools were evaluated, using the quintile analysis; and the constant value of the upper quintile and its respective trend were determined as the goal of each variable. The Chart 2 presents the compiled results, as well as the values observed in each of the variables in the selected schools.

4 Conclusion

In this research, indicators and pedagogical and management goals used in high-performance municipal elementary schools in reading and writing evaluations were identified. Aware of the importance of socioeconomic factors in the performance of students, the authors chose to analyze the practices performed in Brazilian municipal elementary schools of similar socioeconomic level. Indicators related to the various actors of the educational process were found, namely: students, teachers and principals. Of these actors, some were related to the Principal’s management skills, reinforcing his/her role in promoting the quality of teaching.

This article approaches the multiple dimensions of quality pointed out in the literature through social, economic, organizational and institutional performance variables.

The results contribute to show that factors are related to quality of Education, represented here by the concept of school performance; however, other dimensions such as social inequality, teacher Education, management capacity of the principal, pedagogical activities and intra and extra school learning strategies are understood as explanatory factors.

For policymakers, this paper also contributes by presenting a benchmarking of efficient schools. This measure points to a direction for the planning of school principals and education departments.

Results corroborate the importance of the school and the adoption of good pedagogical and management practices in the educational process. In addition, resorting to this study, it is possible for each school to evaluate its own individual performance in relation to each of the indicators related to school performance in reading and writing.

This research is limited to the use of quantitative data for analysis and completion. In addition, the database used to identify explanatory factors comes from declaratory and opinion research, which could have subjectivity in perception. Still, we chose it use it because it is the only database available. Future research should include qualitative data analysis and on-site research on how these practices are performed, thus providing subsidies to the various educational actors for the performance of their functions.

References

Publication Dates

History