Tourism

Introduction

Predicting house prices using a dataset from Kaggle that includes features like the number of bedrooms, bathrooms, living space size, population, and household price. It helps potential buyers understand fair market prices for properties with specific attributes, making it easier to decide whether a property is priced appropriately. Agents can use the model to help price homes accurately, avoiding underpricing or overpricing. Overall, it solves the problem of estimating housing prices, which can assist various stakeholders in making informed financial decisions regarding real estate.

​

With features covering both property-specific and locational attributes, this dataset offers a rich foundation for building a machine learning model capable of understanding the complex relationships that drive property prices. The end goal is to utilize these insights to produce reliable and interpretable price predictions, ultimately enhancing decision-making in real estate investments.

​Regression and How it works

Regression method is utilized to capture a target variable by prediction given one ore more input features. Linear regression

models is as stated a straight-line(linear) comparing the target and the input variables. Basically, finding the best fit line through d data points by minimizing prediction errors.  

​

Linear Regression: (Single feature)

y = B(0) + B(1)x+E

​

• y is the dependent variable (e.g., house price),

• x is the independent variable (e.g., square footage),

• B0​ is the intercept, representing the baseline value of y when x=0

• B1​ is the slope, indicating the change in y for a one-unit increase in x,

• E represents the error term, capturing the variability in y not explained by x.

​

The y is the target and x is the feature, B(0) is the intercept and B(1) is the slope. Linear Regression is basically used for simplicity and interpretability but is based on a linear relationship between the two variables input and output. 

 

Data Understanding and Pre-processing

​I'll be focusing on relevant columns and ignoring irrelevant ones like 'street', 'state zip', 'country', and 'date'. By calculating correlations to see if key features, like square footage and price, are related, as this can guide our predictions. Visualizations, including heatmaps, histograms, and scatter plots, help reveal patterns, such as feature distributions and linear relationships, which lay the groundwork for effective modeling. Provides a clear picture of the data's structure and potential insights before proceeding to model building.

​

Code identifies missing values ​

​df.columns # found 18 columns

​

df = pd.read_csv('datasets/USA Housing Dataset.csv')
# Dropping irrelevant columns
df_cleaned = df.drop(columns=['street', 'statezip', 'country', 'date'])

# Checking for missing values
missing_values = df_cleaned.isnull().sum()

missing_values

​

The 'street', 'statezip','country','data'are irravelent columns that don't require necessary exploration. There are no missing values in the dataset after looking for missing values. So the data is clean considering the results. 

​

​

Experiment 1: Evaluating Performance

Next, I'll encode the city column using one-hot encoding and then split the data into features (X) and target (y) for model training. After that, we can scale the numerical features and proceed to build a linear regression model.
Results:

(54774079827.50566, 0.47752327662060257)

​Results: 

​Mean Squared Error (MSE): 54,828,701,952.41
R-squared (R^(2)): 0.477
47%

​

The R² score indicates that the model explains approximately 47.7% of the variance in housing prices, which leaves room for improvement. The MSE suggests that the average squared difference between predicted and actual prices is quite large, which might imply that additional features or different models could improve performance.
​

​

Experiment 2: 

For Experiment 2, I made changes that might improve the model’s performance. First add interaction terms or polynomial features to capture non-linear relationships between features, especially between square footage and price. Additionally, trying a more complex model like Random Forest Regression or Gradient Boosting could better capture interactions and non-linear patterns that a simple linear regression model may miss.

Alternatively, we could refine our feature selection by testing feature importance or using dimensionality reduction techniques After implementing these adjustments, re-evaluate model performance by comparing the new Mean Squared Error (MSE) and R-squared (R²) values with those from the linear regression model in Experiment 1.

Ideally, we would expect these changes to reduce MSE and increase R², indicating that the new approach captures more variance in housing prices and improves prediction accuracy.

Random Forest Model Evaluation:

Mean Squared Error (MSE): 57042027934.65769

R-squared (R²): 0.45588986717673896

After making predictions, it evaluates the model using Mean Squared Error (MSE) and R-squared (R²) metrics to gauge its performance. This provides insights into how well the Random Forest model fits the housing price data. Unfortuantely the 45.6% is still not tuning the data to our specifications. Further improvement is required to find exactly what we need to get R-Squared to increase. 

​

Experiment 3: ​

Considering the previous models I want to make sure that Gradient Boosting model would perform better than the rest but unfortunately the results have shown a different story. 

​

The evaluation results for the Gradient Boosting model :

• Mean Squared Error (MSE): 108,151,522,201.65

• R-squared (R²): -0.032 (indicating the model performed worse than a simple mean-based prediction)

These results suggest that the Gradient Boosting model is currently underperforming, possibly due to the standard scaling of features, which may not align with Gradient Boosting's requirements. Gradient Boosting often performs better with raw or minimally processed features. Little did I realize that to late considering the dataset. Adjusting preprocessing or fine-tuning model parameters could improve results. ​

 

​Impact Section

This housing price prediction project can make real estate markets more transparent, helping buyers make informed decisions and potentially improving housing affordability. However, it raises ethical concerns, as models trained on historical data could perpetuate pricing biases across neighborhoods, affecting property values in underserved areas. Privacy is another factor, as using detailed property data requires adherence to strict privacy standards. Economically, this model could enhance market efficiency, though it might give larger firms an advantage, potentially impacting fair competition. Careful handling of these aspects is essential to ensure equitable outcomes and minimize unintended harm.

Conclusion​

From this project and the experiments, I learned the importance of data preprocessing and model selection in improving predictive performance. For instance, scaling numerical features helped standardize the data, which is crucial for models sensitive to feature magnitudes, like linear regression. However, we saw that standard scaling may not benefit all models equally, as Gradient Boosting performed worse after scaling, showing that some algorithms handle raw data better.

Additionally, encoding categorical variables using one-hot encoding was essential, as it allowed us to incorporate the 'city' feature effectively without introducing categorical bias. Experimenting with different models, such as Random Forest and Gradient Boosting, highlighted the limitations and strengths of each. Random Forest captured interactions better than linear regression, though it still left room for improvement.

Through these trials, I also realized that choosing relevant features and experimenting with model parameters are crucial steps. Fine-tuning, feature selection, or using ensemble methods could potentially enhance model accuracy. Ultimately, this project emphasized that iterative experimentation with preprocessing and model adjustments is key to refining predictive performance.

​

References

"The Elements of Statistical Learning" by Hastie, Tibshirani, and Friedman​​​

Module Tutorials(in class) 

Gradient Boost Regressor

​https://scikit-learn.org/1.5/modules/generated/sklearn.ensemble.HistGradientBoostingRegressor.html

Kaggle.com

https://xgboost.readthedocs.io/en/latest/

https://scikit-learn.org/stable/

​

For jupyter file please click download

​

​

​

​

https://github.com/Andrad7P/Project-3.git

​