With the vigorous development of cloud computing, most organizations have shifted their data and applications to the cloud environment for storage, computation, and sharing purposes. During storage and data sharing across the participating entities, a malicious agent may gain access to outsourced data from the cloud environment. A malicious agent is an entity that deliberately breaches the data. This information accessed might be misused or revealed to unauthorized parties. Therefore, data protection and prediction of malicious agents have become a demanding task that needs to be addressed appropriately. To deal with this crucial and challenging issue, this paper presents a Malicious Agent Identification-based Data Security (MAIDS) Model which utilizes XGBoost machine learning classification algorithm for securing data allocation and communication among different participating entities in the cloud system. The proposed model explores and computes intended multiple security parameters associated with online data communication or transactions. Correspondingly, a security-focused knowledge database is produced for developing the XGBoost Classifier-based Malicious Agent Prediction (XC-MAP) unit. Unlike the existing approaches, which only identify malicious agents after data leaks, MAIDS proactively identifies malicious agents by examining their eligibility for respective data access. In this way, the model provides a comprehensive solution to safeguard crucial data from both intentional and non-intentional breaches, by granting data to authorized agents only by evaluating the agent’s behavior and predicting the malicious agent before granting data. The performance of the proposed model is thoroughly evaluated by accomplishing extensive experiments, and the results signify that the MAIDS model predicts the malicious agents with high accuracy, precision, recall, and F1-scores up to 95.55%, 95.30%, 95.50%, and 95.20%, respectively. This enormously enhances the system’s security in terms of authorized data access accuracy up to 55.49%, precision up to 43.15%, recall up to 55.49%, and F1-score up to 39.96%, respectively, as compared to state-of-the-art work.