blah blah blah is here! blah blah » Close

up1down
link

I am developing a ppc application that needs to synchronize its database .sdf with some other sql data staying in a server. Ok the sdf i think is nice for having a database in a ppc but is really slow, specially in the insert... and with active sync can't do so much cause it works with only one pocket at time... so i am doing:

- call a socket client(ppc)/server(pc)
- make some choices from the ppc app for preparing some specific rows
of a specific database; send as request to the server
- the server creates programmatically databases .sdf with what client needs and give to _all_ the clients a response
- each client when gets its own response copies this file from the server...

it works better cause i've tried with simple sql select and insert from ppc to the pc and it was 12 000 rows -> 55~60 sec and is toooooo much ... while this way 3~4 sec to server for preparing the database, other 3~4 to client for getting it... so not bad...

but now i don't know... if i make a call that has to prepare too much rows from the database and while creating this database, to the socket cames another request for preparing another database for a second ppc... what can happen? So do i have to make some threads for each process? ..that i suppose yea...
and the other thing do you think is the right way or maybe i could use some other directions?

Thanks
Daniel

...i wanted to make the server more interactive... putting a table with some code_requests corresponding to sql queries... and to each type of queries a specific type of database file... so i don't know if this kind of socket app is robust for processing different calls... i've followed this link: http://www.codeguru.com/csharp/csharp/cs_network/sockets/article.php/c7695 for the socket.

last answered one year ago

4 answers

up0down
link

Ok here is how I would implement this solution.

1. Create the remote database
- I create the remove database. optimize it as much as I can.
1.1. Column indexing : anything you would use in a WHERE clause. index it.
1.2. view creation : For read-only purposes...common data that would be access regularly.
since this application will be distributed to clients...there will be a lot of reading to the main database and having views seriously speeds things up.
1.3. stored procedures. The less data I have to send to the server. the better.

2. Creating the applications local data store
2.1. I would XML Database ( heard of xpath ? ).
note : XML is the fastest and most compatible way to move data around.
note : the XML schema needs to be a replication of that your main database used.
note : to keep things secure you could access a page such as .aspx or .php to act as a proxy between your xml data files and application. you can then serve these files without any anon user knowing it's actual path.

3. building the application and making it work with your local data store
insert, update, delete, select, create etc

4. updating the main data store.
I have 3 ways for this.
4.1. windows service.
if you have direct access to server. you can write and install a windows service that will update your main data store every 8 hours or so.

4.2. SQL Server import data.
if you have access to the SQL server. you can use built in functionality to IMPORT the data.
here is an example of what you can do : with xml to/from SQL.
http://msdn.microsoft.com/en-us/library/ms191184.aspx

4.3. update on request if expired.
you create a text file on the server which will store a time value of your last update to the main db. each time you application is accessed you read this time value. if the time value falls within an 8 hour period or more. the application will update the DB ( not really recommended )

Advantages :
using XML as your local data store would - in most cases -give near split second query processing. ( yes even with your 12000 rows )
updating the remote location as a batch would be better as SQL Server supports batch execution nicely.
see here :
http://www.teratrax.com/tdm/help/queries.html

up0down
link

Thanks A_O
at the end:

- have a server app that takes some args
- call that from client ppc using: EXEC master..xp_cmdshell @cmd and passing what i was passing with the socket (in this way i don't have any more the problem of multiple calls)
and replay back a confirm with the url of a created sdf from the server... in a wamp folder
- the client downloads from that address the database...

tested: 50 000 rows ~ 22-25 sec

the only thing that complicates a little bit the situation is that i can't enable the filesharing on windows ce... i wasn't able at least... and so i have to do this thing of putting that file in wamp but i think in this way can synchronize from every were having a wireless connection.

But i want to see and to try xml too..

Anyway thanks again for the idea..

Daniel

you said 50 000 rows in 22.25 seconds. can you show me your Query and if possible table structure. I could help you optimize it. but if you are distributing the system to many ppl. XML is going to be better for performance.

up0down
link

yea 22.25 sec but for server to prepare a sdf file and to the client pocket pc to download the sdf file...
and when i tried with direct connection with the sql server, writing down to the sdf database with simple insert, it took 55.60 sec to get 12 000 rows...

//For creating the database file

string connectionString = @"DataSource='c:/Debug/" + dbname + ".sdf'; Password=''";
SqlCeEngine en = new SqlCeEngine(connectionString);
en.CreateDatabase();

SqlCeConnection conn = new SqlCeConnection(@"Data Source=C:/Debug/" + dbname + ".sdf;Password=''");
conn.Open();
string query = "create table art ( bc nvarchar (40) not null, scrn nvarchar (100) not null)";

SqlCeCommand cmd = new SqlCeCommand(query, conn);
cmd.ExecuteNonQuery();



// for filling it..

SqlCeConnection conn = new SqlCeConnection(@"Data Source='c:/Debug/" + dbname + ".sdf'; Password=''");


conn.Open();
string Server = "erss";
string Username = "Sa";
string Password = "";
string Database = dbname;

string ConnectionString = "Data Source=" + Server + ";";
ConnectionString += "User ID=" + Username + ";";
ConnectionString += "Password=" + Password + ";";
ConnectionString += "Initial Catalog=" + Database + ";";


SqlConnection SQLConnection = new SqlConnection();
try
{

SqlDataReader myReader = null;
SQLConnection.ConnectionString = ConnectionString;
SQLConnection.Open();

SqlCeCommand cmd = conn.CreateCommand();

string strSQLCommand = "select bc, scrn from art where bc is not null and scrn is not null";

SqlCommand command = new SqlCommand(strSQLCommand, SQLConnection);
myReader = command.ExecuteReader();



while (myReader.Read())
{
string code = (string)myReader["bc"];
string scrn= (string)myReader["scrn"];

cmd.Parameters.Clear();
cmd.CommandText = "INSERT INTO art (bc, scrn) Values(@code, @scrn)";

cmd.Parameters.AddWithValue("@code", code);
cmd.Parameters.AddWithValue("@scrn", scrn);
cmd.ExecuteNonQuery();

}


myReader.Close();
conn.Close();
SQLConnection.Close();
string from = @"C:/Debug/" + dbname + ".sdf";
string to =@"C:/wamp/www/dbsppc/" + dbname + ".sdf";

File.Move(from, to);
Console.WriteLine("OK");


thats all call this from client with xp_cmdshell giving dhe database name and a userid for some authentication
...
if all this completes it will turn back ok..... the client downloads the file.

there are other 2 tables but for configuration...

up0down
link

lets look at this code of yours.

//For creating the database file

SqlCeEngine en = new SqlCeEngine(connectionString);
en.CreateDatabase();

SqlCeConnection conn = new SqlCeConnection(@"Data Source=C:/Debug/" + dbname + ".sdf;Password=''");
conn.Open();

it seems to me you are creating 2 connections to your db ? are you closing or disposing them anywhere ?
if you can put both these tasks into a single connection that would definitely help as the port will not be shared between 2 of your objects.
check this example here when working with the SqlCeEngine
http://msdn.microsoft.com/en-us/library/system.data.sqlserverce.sqlceengine(VS.80).aspx

What you can do is :

1. can you show me the data structure of your application : tables, views, stored procedures
From there I can help you optimize queries.

on another note :
I also noticed your are hard coding your queries in your application.
if you make them all as stored procedures and just pass data to them. you send less to the DB
example of using a stored proc
string query_sp = "exec sp_insert_art @code, @scrn";
// compare this to your current which is
// you sending about 50% less data to the server already when using a stored proc
// you also don't need to change the rest of your code...just the query line
string query_hard = "INSERT INTO art (bc, scrn) Values(@code, @scrn)";
/// your command text would be
cmd.CommandText = query_sp;

// also looking at this
string strSQLCommand = "select bc, scrn from art where bc is not null and scrn is not null";
//this could become
string strSQLCommand = "exec sp_getmydata";


Remember on the DB Side : INDEXING and VIEWS work wonders on huge databases.

Feedback