Tuesday 13 December 2022

Statistical fundamentals and terminology for model building and validation

1. Mean: This is a simple arithmetic average, which is computed by taking the aggregated sum of values divided by a count of those values. The mean is sensitive to outliers in the data. An outlier is the value of a set or column that is highly deviant from the many other values in the same data; it usually has very high or low values. 

2. Median: This is the midpoint of the data, and is calculated by either arranging it in ascending or descending order. If there are N observations. 

3. Mode: This is the most repetitive data point in the data


import numpy as np
import statistics as stats
data = np.array([4,5,1,2,7,2,6,9,3,9,9,2])
# Calculate Mean
dt_mean = np.mean(data) ; print ("Mean :",round(dt_mean,2))
# Calculate Median
dt_median = np.median(data) ; print ("Median :",dt_median)
# Calculate Mode
dt_mode = stats.multimode(data);
print(dt_mode)

Result:
Mean : 4.92 Median : 4.5 Mode: [2, 9]

Monday 6 June 2022

Python grouping

 import numpy as np

import pandas as pd
from pandas import Series, DataFrame
import seaborn as sn

address = 'mtcars.csv'
cars = pd.read_csv(address)
cars.columns = ['car_names', 'mpg', 'cyl', 'disp', 'hp', 'drat', 'wt', 'qsec', 'vs', 'am', 'gear', 'carb']
cars.head()
address = 'mtcars.csv'

cars = pd.read_csv(address)

cars.columns = ['car_names', 'mpg', 'cyl', 'disp', 'hp', 'drat', 'wt', 'qsec', 'vs',
                'am', 'gear', 'carb']
cars.head()
cars_groups = cars.groupby(cars['cyl'])
#cars_groups.mean()
sn.barplot(x='cyl', y = 'disp', data = cars)
cars['cyl'].value_counts().plot.bar();
cars['gear'].value_counts().plot.bar();
cars_groups = cars.groupby(cars['gear'])
cars_groups.mean()
cars_groups = cars.groupby(cars['gear'])
cars_groups.count()

Python read json data from API

import pandas as pd

import requests


d = requests.get("https://www.yourwebsite.com/controller?ApiKey=value").json()
df = pd.DataFrame.from_dict(d['records'])
df.shape

Tuesday 19 April 2022

How to transpose data in MSSQL

 

Declare @UserID nvarchar(256)

Declare       @MsgType int

 

set @UserID = userID

set @MsgType = 3

 

CREATE TABLE #tmpBus3

(

   AdminUnitID int,

   AdminUnitName nvarchar(256),

   CategoryName nvarchar(256),

   CategoryID int,

   CategoryLevelID int,

   ParentID int,

   IsMainAdminUnit bit,

   PersonID int,

   Name nvarchar(256),

   UserID nvarchar(256),

   level int,

   LevelName nvarchar(256)

)

       INSERT INTO #tmpBus3

       Exec [Archv].Cust_GetContactsbyUserID @UserID,@MsgType;

 

       DECLARE @tmpTable3 AS nvarchar(max);

       SELECT @tmpTable3 = CONCAT(COALESCE(@tmpTable3 + ']  nvarchar(255), [','['),CONCAT('Col', ROW_NUMBER() OVER(ORDER BY LevelName ASC)))

         from #tmpBus3

         set @tmpTable3='CREATE TABLE #tmpTable3 ( '+@tmpTable3+']  nvarchar(255) )'

           DECLARE @levels AS nvarchar(max);

SELECT @levels = CONCAT(COALESCE(+@levels + ''',N''',''), AdminUnitName)

  from #tmpBus3

  set @levels=''''+@levels+''''

 

         DECLARE @SqlStatement1 NVARCHAR(MAX)

         SET @SqlStatement1 = N''+@tmpTable3+' insert into #tmpTable3 values ('+@levels+') select * from #tmpTable3';     

         print @SqlStatement1

          EXEC(@SqlStatement1)

Drop Table #tmpBus3

The result will be as in the following screenshot




Tuesday 22 March 2022

Enable geoserver to serve frond end and overcome the CORS problem

 Go to geoserver\lib in the lib directory of geoserve to find jetty-servlets-XX.jar and jetty-util-xx.jar (xx means different versions) two jar packages


Copy the all jars packagesfrom geoserver\lib to the geoserver\webapps\geoserver\WEB-INF\lib\ directory

 Go to the geoserver\webapps\geoserver\WEB-INF directory to find web.xml and add the following XML:


<context-param>

    <param-name>ENABLE_JSONP</param-name>

    <param-value>true</param-value>

</context-param>

<filter>

      <filter-name>cross-origin</filter-name>

      <filter-class>org.eclipse.jetty.servlets.CrossOriginFilter</filter-class>

  </filter>

   <filter-mapping>

        <filter-name>cross-origin</filter-name>

        <url-pattern>/*</url-pattern>

    </filter-mapping>



add the following line at the beginning of startup file of geoserver   

"C:\Program Files\GeoServer\bin\startup.bat" edit it by notepad++

set geoserver.xframe.shouldSetPolicy=false


or using the service 

open the wrapper folder C:\Program Files\GeoServer\wrapper

and edit the cmd command which placed at the end of the jsl64 file

-Dgeoserver.xframe.shouldSetPolicy=false



stop the service and run the startup.bat as administrator

Everything will be OK


Thursday 3 February 2022

DNN delete all duplicated users

declare @email as nvarchar(256)

 

DECLARE db_cursor CURSOR FOR

SELECT [Email]

  FROM [Users]

  group by [Email] 

  having count([Email])>1

 

OPEN db_cursor 

FETCH NEXT FROM db_cursor INTO @email 

 

WHILE @@FETCH_STATUS = 0 

BEGIN 

    

         UPDATE dbo.UserPortals

                           SET

                                  IsDeleted = 1

                           WHERE  PortalId = 0

                           and UserId in (

SELECT [UserID]

  FROM [Users]

  where Email =@email and ([UserID] not in (SELECT [UserID]

  FROM [RelatedTable1]

  where [UserID] in(SELECT [UserID]

  FROM [Users]

  where Email =@email)

  UNION

  SELECT [UserID]

  FROM [RelatedTable2]

  where [UserID] in(SELECT [UserID]

  FROM [Users]

  where Email =@email))))

      FETCH NEXT FROM db_cursor INTO @email

END

 

CLOSE db_cursor 

DEALLOCATE db_cursor


Wednesday 12 January 2022

globals.navigateurl is obsolete

 

DotNetNuke.Common.Globals.NavigateURL() in Dnn 8, but in DNN 9 use the following 


  • PortalSettings.ActiveTab.FullUrl to replace NavigateURL()
  • Response.Redirect(EditURL()) to replace NavigateURL() with parameters